In hierarchical forecasting, we aim to create forecasts for many time series concurrently, whilst adhering to pre-specified hierarchical relationships that exist between the time series. We can enforce this coherence by performing a post-processing reconciliation step on the forecasts.

The HierarchicalForecast package provides the most comprehensive collection of Python implementations of hierarchical forecasting algorithms that follow classic hierarchical reconciliation. All the methods have a reconcile function capable of reconciling base forecasts using numpy arrays.

​

Cross-sectional hierarchies

Traditionally, hierarchical forecasting methods reconcile cross-sectional aggregations. For example, we may have forecasts for individual product demand, but also for the overall product group, department and store, and we are interested in making sure these forecasts are coherent with each other. This can be formalized as:

$\tilde{\textbf{Y}} = SP\hat{\textbf{Y}} \;,$

where $\hat{\textbf{Y}} \in \mathbb{R}^{m \times p}$ denotes the matrix of forecasts for all $m$ time series for all $p$ time steps in the hierarchy, $S \in \lbrace 0, 1 \rbrace^{m \times n}$ is a matrix that defines the hierarchical relationship between the $n$ bottom-level time series and the $m^* = m - n$ aggregations, $P \in \mathbb{R}^{n \times m}$ is a matrix that encapsulates the contribution of each forecast to the final estimate, and $\tilde{\textbf{Y}} \in \mathbb{R}^{m \times p}$ is the matrix of reconciled forecasts. We can use the matrix $P$ to define various forecast contribution scenarios.

Cross-sectional reconciliation methods aim to find the optimal $P$ matrix.

​

Temporal hierarchies

We can also perform temporal reconciliation. For example, we may have forecasts for daily demand, weekly, and monthly, and we are interested in making sure these forecasts are coherent with each other. We formalize the temporal hierarchical forecasting problem as:

$\tilde{\textbf{Y}} = \left( S_{te} P_{te} \hat{\textbf{Y}}^{\intercal} \right)^{\intercal} \;,$

where $S_{te} \in \lbrace 0, 1 \rbrace^{p \times k}$ is a matrix that defines the hierarchical relationship between the $k$ bottom-level time steps and the $p^* = p - k$ aggregations and $P_{te} \in \mathbb{R}^{k \times p}$ is a matrix that encapsulates the contribution of each forecast to the final estimate. We can use the matrix $P_{te}$ to define various forecast contribution scenarios.

Temporal reconciliation methods aim to find the optimal $P_{te}$ matrix.

​

Cross-temporal reconciliation

We can combine cross-sectional and temporal hierarchical forecasting by performing cross-sectional reconciliation and temporal reconciliation in a two-step procedure.

References
-Hyndman, Rob. Notation for forecast reconciliation.

​

1. Bottom-Up

​

BottomUp

 BottomUp ()

*Bottom Up Reconciliation Class. The most basic hierarchical reconciliation is performed using an Bottom-Up strategy. It was proposed for the first time by Orcutt in 1968. The corresponding hierarchical “projection” matrix is defined as: $\mathbf{P}_{\text{BU}} = [\mathbf{0}_{\mathrm{[b],[a]}}\;|\;\mathbf{I}_{\mathrm{[b][b]}}]$

Parameters:
None

References:
- Orcutt, G.H., Watts, H.W., & Edwards, J.B.(1968). “Data aggregation and information loss”. The American Economic Review, 58 , 773(787).*

​

BottomUp.fit

 BottomUp.fit (S:numpy.ndarray, y_hat:numpy.ndarray,
               idx_bottom:numpy.ndarray,
               y_insample:Optional[numpy.ndarray]=None,
               y_hat_insample:Optional[numpy.ndarray]=None,
               sigmah:Optional[numpy.ndarray]=None,
               intervals_method:Optional[str]=None,
               num_samples:Optional[int]=None, seed:Optional[int]=None,
               tags:Optional[dict[str,numpy.ndarray]]=None)

*Bottom Up Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: In-sample values of size (base, horizon).
y_hat_insample: In-sample forecast values of size (base, horizon).
sigmah: Estimated standard deviation of the conditional marginal distribution.

intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
**sampler_kwargs: Coherent sampler instantiation arguments.

Returns:
self: object, fitted reconciler.*

​

BottomUp.predict

 BottomUp.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                   level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

BottomUp.fit_predict

 BottomUp.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                       idx_bottom:numpy.ndarray,
                       y_insample:Optional[numpy.ndarray]=None,
                       y_hat_insample:Optional[numpy.ndarray]=None,
                       sigmah:Optional[numpy.ndarray]=None,
                       level:Optional[list[int]]=None,
                       intervals_method:Optional[str]=None,
                       num_samples:Optional[int]=None,
                       seed:Optional[int]=None,
                       tags:Optional[dict[str,numpy.ndarray]]=None)

*BottomUp Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: In-sample values of size (base, insample_size).
y_hat_insample: In-sample forecast values of size (base, insample_size).
sigmah: Estimated standard deviation of the conditional marginal distribution.

level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
**sampler_kwargs: Coherent sampler instantiation arguments.

Returns:
y_tilde: Reconciliated y_hat using the Bottom Up approach.*

​

BottomUp.sample

 BottomUp.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

BottomUpSparse

 BottomUpSparse ()

*BottomUpSparse Reconciliation Class.

This is the implementation of a Bottom Up reconciliation using the sparse matrix approach. It works much more efficient on datasets with many time series. [makoren: At least I hope so, I only checked up until ~20k time series, and there’s no real improvement, it would be great to check for smth like 1M time series, where the dense S matrix really stops fitting in memory]

See the parent class for more details.*

​

BottomUpSparse.fit

 BottomUpSparse.fit (S:numpy.ndarray, y_hat:numpy.ndarray,
                     idx_bottom:numpy.ndarray,
                     y_insample:Optional[numpy.ndarray]=None,
                     y_hat_insample:Optional[numpy.ndarray]=None,
                     sigmah:Optional[numpy.ndarray]=None,
                     intervals_method:Optional[str]=None,
                     num_samples:Optional[int]=None,
                     seed:Optional[int]=None,
                     tags:Optional[dict[str,numpy.ndarray]]=None)

*Bottom Up Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: In-sample values of size (base, horizon).
y_hat_insample: In-sample forecast values of size (base, horizon).
sigmah: Estimated standard deviation of the conditional marginal distribution.

intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
**sampler_kwargs: Coherent sampler instantiation arguments.

Returns:
self: object, fitted reconciler.*

​

BottomUpSparse.predict

 BottomUpSparse.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                         level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

BottomUpSparse.fit_predict

 BottomUpSparse.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                             idx_bottom:numpy.ndarray,
                             y_insample:Optional[numpy.ndarray]=None,
                             y_hat_insample:Optional[numpy.ndarray]=None,
                             sigmah:Optional[numpy.ndarray]=None,
                             level:Optional[list[int]]=None,
                             intervals_method:Optional[str]=None,
                             num_samples:Optional[int]=None,
                             seed:Optional[int]=None,
                             tags:Optional[dict[str,numpy.ndarray]]=None)

*BottomUp Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: In-sample values of size (base, insample_size).
y_hat_insample: In-sample forecast values of size (base, insample_size).
sigmah: Estimated standard deviation of the conditional marginal distribution.

level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
**sampler_kwargs: Coherent sampler instantiation arguments.

Returns:
y_tilde: Reconciliated y_hat using the Bottom Up approach.*

​

BottomUpSparse.sample

 BottomUpSparse.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

2. Top-Down

​

TopDown

 TopDown (method:str)

*Top Down Reconciliation Class.

The Top Down hierarchical reconciliation method, distributes the total aggregate predictions and decomposes it down the hierarchy using proportions $\mathbf{p}_{\mathrm{[b]}}$ that can be actual historical values or estimated.

$\mathbf{P}=[\mathbf{p}_{\mathrm{[b]}}\;|\;\mathbf{0}_{\mathrm{[b][a,b\;-1]}}]$ Parameters:
method: One of forecast_proportions, average_proportions and proportion_averages.

References:
- CW. Gross (1990). “Disaggregation methods to expedite product line forecasting”. Journal of Forecasting, 9 , 233–254. doi:10.1002/for.3980090304.
- G. Fliedner (1999). “An investigation of aggregate variable time series forecast strategies with specific subaggregate time series statistical correlation”. Computers and Operations Research, 26 , 1133–1149. doi:10.1016/S0305-0548(99)00017-9.*

​

TopDown.fit

 TopDown.fit (S, y_hat, y_insample:numpy.ndarray,
              y_hat_insample:Optional[numpy.ndarray]=None,
              sigmah:Optional[numpy.ndarray]=None,
              intervals_method:Optional[str]=None,
              num_samples:Optional[int]=None, seed:Optional[int]=None,
              tags:Optional[dict[str,numpy.ndarray]]=None,
              idx_bottom:Optional[numpy.ndarray]=None)

*TopDown Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
y_insample: Insample values of size (base, insample_size). Optional for forecast_proportions method.
y_hat_insample: Insample forecast values of size (base, insample_size). Optional for forecast_proportions method.
sigmah: Estimated standard deviation of the conditional marginal distribution.
interval_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).

Returns:
self: object, fitted reconciler.*

​

TopDown.predict

 TopDown.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                  level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

TopDown.fit_predict

 TopDown.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                      tags:dict[str,numpy.ndarray],
                      idx_bottom:numpy.ndarray=None,
                      y_insample:Optional[numpy.ndarray]=None,
                      y_hat_insample:Optional[numpy.ndarray]=None,
                      sigmah:Optional[numpy.ndarray]=None,
                      level:Optional[list[int]]=None,
                      intervals_method:Optional[str]=None,
                      num_samples:Optional[int]=None,
                      seed:Optional[int]=None)

*Top Down Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: Insample values of size (base, insample_size). Optional for forecast_proportions method.
y_hat_insample: Insample forecast values of size (base, insample_size). Optional for forecast_proportions method.
sigmah: Estimated standard deviation of the conditional marginal distribution.
level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.

Returns:
y_tilde: Reconciliated y_hat using the Top Down approach.*

​

TopDown.sample

 TopDown.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

TopDownSparse

 TopDownSparse (method:str)

*TopDownSparse Reconciliation Class.

This is an implementation of top-down reconciliation using the sparse matrix approach. It works much more efficiently on data sets with many time series.

See the parent class for more details.*

​

TopDownSparse.fit

 TopDownSparse.fit (S, y_hat, y_insample:numpy.ndarray,
                    y_hat_insample:Optional[numpy.ndarray]=None,
                    sigmah:Optional[numpy.ndarray]=None,
                    intervals_method:Optional[str]=None,
                    num_samples:Optional[int]=None,
                    seed:Optional[int]=None,
                    tags:Optional[dict[str,numpy.ndarray]]=None,
                    idx_bottom:Optional[numpy.ndarray]=None)

*TopDown Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
y_insample: Insample values of size (base, insample_size). Optional for forecast_proportions method.
y_hat_insample: Insample forecast values of size (base, insample_size). Optional for forecast_proportions method.
sigmah: Estimated standard deviation of the conditional marginal distribution.
interval_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).

Returns:
self: object, fitted reconciler.*

​

TopDownSparse.predict

 TopDownSparse.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                        level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

TopDownSparse.fit_predict

 TopDownSparse.fit_predict (S:scipy.sparse._csr.csr_matrix,
                            y_hat:numpy.ndarray,
                            tags:dict[str,numpy.ndarray],
                            idx_bottom:numpy.ndarray=None,
                            y_insample:Optional[numpy.ndarray]=None,
                            y_hat_insample:Optional[numpy.ndarray]=None,
                            sigmah:Optional[numpy.ndarray]=None,
                            level:Optional[list[int]]=None,
                            intervals_method:Optional[str]=None,
                            num_samples:Optional[int]=None,
                            seed:Optional[int]=None)

*Top Down Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: Insample values of size (base, insample_size). Optional for forecast_proportions method.
y_hat_insample: Insample forecast values of size (base, insample_size). Optional for forecast_proportions method.
sigmah: Estimated standard deviation of the conditional marginal distribution.
level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.

Returns:
y_tilde: Reconciliated y_hat using the Top Down approach.*

​

TopDownSparse.sample

 TopDownSparse.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

cls_top_down(
                S=S, y_hat=S @ y_hat_bottom, y_insample=S @ y_bottom, tags=tags
            )["mean"]

#\ hide
cls_top_down = TopDownSparse(method="average_proportions")
test_fail(
    cls_top_down,
    contains="Top-down reconciliation requires strictly hierarchical structures.",
    args=(sparse.csr_matrix(S_non_hier), None, tags_non_hier),
)

​

3. Middle-Out

​

MiddleOut

 MiddleOut (middle_level:str, top_down_method:str)

*Middle Out Reconciliation Class.

This method is only available for strictly hierarchical structures. It anchors the base predictions in a middle level. The levels above the base predictions use the Bottom-Up approach, while the levels below use a Top-Down.

Parameters:
middle_level: Middle level.
top_down_method: One of forecast_proportions, average_proportions and proportion_averages.

References:
- Hyndman, R.J., & Athanasopoulos, G. (2021). “Forecasting: principles and practice, 3rd edition: Chapter 11: Forecasting hierarchical and grouped series.”. OTexts: Melbourne, Australia. OTexts.com/fpp3 Accessed on July 2022.*

​

MiddleOut.fit

 MiddleOut.fit (**kwargs)

​

MiddleOut.predict

 MiddleOut.predict (**kwargs)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

MiddleOut.fit_predict

 MiddleOut.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                        tags:dict[str,numpy.ndarray],
                        y_insample:Optional[numpy.ndarray]=None,
                        level:Optional[list[int]]=None,
                        intervals_method:Optional[str]=None)

*Middle Out Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
tags: Each key is a level and each value its S indices.
y_insample: Insample values of size (base, insample_size). Only used for forecast_proportions
level: Not supported.
intervals_method: Not supported.

Returns:
y_tilde: Reconciliated y_hat using the Middle Out approach.*

​

MiddleOut.sample

 MiddleOut.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

MiddleOutSparse

 MiddleOutSparse (middle_level:str, top_down_method:str)

*MiddleOutSparse Reconciliation Class.

This is an implementation of middle-out reconciliation using the sparse matrix approach. It works much more efficiently on data sets with many time series.

See the parent class for more details.*

​

MiddleOutSparse.fit

 MiddleOutSparse.fit (**kwargs)

​

MiddleOutSparse.predict

 MiddleOutSparse.predict (**kwargs)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

MiddleOutSparse.fit_predict

 MiddleOutSparse.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                              tags:dict[str,numpy.ndarray],
                              y_insample:Optional[numpy.ndarray]=None,
                              level:Optional[list[int]]=None,
                              intervals_method:Optional[str]=None)

*Middle Out Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
tags: Each key is a level and each value its S indices.
y_insample: Insample values of size (base, insample_size). Only used for forecast_proportions
level: Not supported.
intervals_method: Not supported.

Returns:
y_tilde: Reconciliated y_hat using the Middle Out approach.*

​

MiddleOutSparse.sample

 MiddleOutSparse.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

4. Min-Trace

​

MinTrace

 MinTrace (method:str, nonnegative:bool=False,
           mint_shr_ridge:Optional[float]=2e-08, num_threads:int=1)

*MinTrace Reconciliation Class.

This reconciliation algorithm proposed by Wickramasuriya et al. depends on a generalized least squares estimator and an estimator of the covariance matrix of the coherency errors $\mathbf{W}_{h}$. The Min Trace algorithm minimizes the squared errors for the coherent forecasts under an unbiasedness assumption; the solution has a closed form.

Parameters:
method: str, one of ols, wls_struct, wls_var, mint_shrink, mint_cov.
nonnegative: bool, reconciled forecasts should be nonnegative?
mint_shr_ridge: float=2e-8, ridge numeric protection to MinTrace-shr covariance estimator.
num_threads: int=1, number of threads to use for solving the optimization problems (when nonnegative=True).

References:
- Wickramasuriya, S. L., Athanasopoulos, G., & Hyndman, R. J. (2019). “Optimal forecast reconciliation for hierarchical and grouped time series through trace minimization”. Journal of the American Statistical Association, 114 , 804–819. doi:10.1080/01621459.2018.1448825.. - Wickramasuriya, S.L., Turlach, B.A. & Hyndman, R.J. (2020). “Optimal non-negative forecast reconciliation”. Stat Comput 30, 1167–1182, https://doi.org/10.1007/s11222-020-09930-0.*

​

MinTrace.fit

 MinTrace.fit (S, y_hat, y_insample:Optional[numpy.ndarray]=None,
               y_hat_insample:Optional[numpy.ndarray]=None,
               sigmah:Optional[numpy.ndarray]=None,
               intervals_method:Optional[str]=None,
               num_samples:Optional[int]=None, seed:Optional[int]=None,
               tags:Optional[dict[str,numpy.ndarray]]=None,
               idx_bottom:Optional[numpy.ndarray]=None)

*MinTrace Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
y_insample: Insample values of size (base, insample_size). Only used with “wls_var”, “mint_cov”, “mint_shrink”.
y_hat_insample: Insample forecast values of size (base, insample_size). Only used with “wls_var”, “mint_cov”, “mint_shrink”
sigmah: Estimated standard deviation of the conditional marginal distribution.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).

Returns:
self: object, fitted reconciler.*

​

MinTrace.predict

 MinTrace.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                   level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

MinTrace.fit_predict

 MinTrace.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                       idx_bottom:numpy.ndarray=None,
                       y_insample:Optional[numpy.ndarray]=None,
                       y_hat_insample:Optional[numpy.ndarray]=None,
                       sigmah:Optional[numpy.ndarray]=None,
                       level:Optional[list[int]]=None,
                       intervals_method:Optional[str]=None,
                       num_samples:Optional[int]=None,
                       seed:Optional[int]=None,
                       tags:Optional[dict[str,numpy.ndarray]]=None)

*MinTrace Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: Insample values of size (base, insample_size). Only used by wls_var, mint_cov, mint_shrink
y_hat_insample: Insample fitted values of size (base, insample_size). Only used by wls_var, mint_cov, mint_shrink
sigmah: Estimated standard deviation of the conditional marginal distribution.
level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.

Returns:
y_tilde: Reconciliated y_hat using the MinTrace approach.*

​

MinTrace.sample

 MinTrace.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

MinTraceSparse

 MinTraceSparse (method:str, nonnegative:bool=False, num_threads:int=1,
                 qp:bool=True)

*MinTraceSparse Reconciliation Class.

This is the implementation of OLS and WLS estimators using sparse matrices. It is not guaranteed to give identical results to the non-sparse version, but works much more efficiently on data sets with many time series.

See the parent class for more details.

Parameters:
method: str, one of ols, wls_struct, or wls_var.
nonnegative: bool, return non-negative reconciled forecasts.
num_threads: int, number of threads to execute non-negative quadratic programming calls.
qp: bool, implement non-negativity constraint with a quadratic programming approach. Setting this to True generally gives better results, but at the expense of higher cost to compute.
*

​

MinTraceSparse.fit

 MinTraceSparse.fit (S:scipy.sparse._csr.csr_matrix, y_hat:numpy.ndarray,
                     y_insample:Optional[numpy.ndarray]=None,
                     y_hat_insample:Optional[numpy.ndarray]=None,
                     sigmah:Optional[numpy.ndarray]=None,
                     intervals_method:Optional[str]=None,
                     num_samples:Optional[int]=None,
                     seed:Optional[int]=None,
                     tags:Optional[dict[str,numpy.ndarray]]=None,
                     idx_bottom:Optional[numpy.ndarray]=None)

*MinTraceSparse Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
y_insample: Insample values of size (base, insample_size). Only used with “wls_var”.
y_hat_insample: Insample forecast values of size (base, insample_size). Only used with “wls_var”
sigmah: Estimated standard deviation of the conditional marginal distribution.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).

Returns:
self: object, fitted reconciler.*

​

MinTraceSparse.predict

 MinTraceSparse.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                         level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

MinTraceSparse.fit_predict

 MinTraceSparse.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                             idx_bottom:numpy.ndarray=None,
                             y_insample:Optional[numpy.ndarray]=None,
                             y_hat_insample:Optional[numpy.ndarray]=None,
                             sigmah:Optional[numpy.ndarray]=None,
                             level:Optional[list[int]]=None,
                             intervals_method:Optional[str]=None,
                             num_samples:Optional[int]=None,
                             seed:Optional[int]=None,
                             tags:Optional[dict[str,numpy.ndarray]]=None)

*MinTrace Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: Insample values of size (base, insample_size). Only used by wls_var, mint_cov, mint_shrink
y_hat_insample: Insample fitted values of size (base, insample_size). Only used by wls_var, mint_cov, mint_shrink
sigmah: Estimated standard deviation of the conditional marginal distribution.
level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.

Returns:
y_tilde: Reconciliated y_hat using the MinTrace approach.*

​

MinTraceSparse.sample

 MinTraceSparse.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

5. Optimal Combination

​

OptimalCombination

 OptimalCombination (method:str, nonnegative:bool=False,
                     num_threads:int=1)

*Optimal Combination Reconciliation Class.

This reconciliation algorithm was proposed by Hyndman et al. 2011, the method uses generalized least squares estimator using the coherency errors covariance matrix. Consider the covariance of the base forecast $\textrm{Var}(\epsilon_{h}) = \Sigma_{h}$, the $\mathbf{P}$ matrix of this method is defined by: $\mathbf{P} = \left(\mathbf{S}^{\intercal}\Sigma_{h}^{\dagger}\mathbf{S}\right)^{-1}\mathbf{S}^{\intercal}\Sigma^{\dagger}_{h}$ where $\Sigma_{h}^{\dagger}$ denotes the variance pseudo-inverse. The method was later proven equivalent to MinTrace variants.

Parameters:
method: str, allowed optimal combination methods: ‘ols’, ‘wls_struct’.
nonnegative: bool, reconciled forecasts should be nonnegative?

References:
- Rob J. Hyndman, Roman A. Ahmed, George Athanasopoulos, Han Lin Shang (2010). “Optimal Combination Forecasts for Hierarchical Time Series”..
- Shanika L. Wickramasuriya, George Athanasopoulos and Rob J. Hyndman (2010). “Optimal Combination Forecasts for Hierarchical Time Series”.. - Wickramasuriya, S.L., Turlach, B.A. & Hyndman, R.J. (2020). “Optimal non-negative forecast reconciliation”. Stat Comput 30, 1167–1182, https://doi.org/10.1007/s11222-020-09930-0.*

​

OptimalCombination.fit

 OptimalCombination.fit (S, y_hat,
                         y_insample:Optional[numpy.ndarray]=None,
                         y_hat_insample:Optional[numpy.ndarray]=None,
                         sigmah:Optional[numpy.ndarray]=None,
                         intervals_method:Optional[str]=None,
                         num_samples:Optional[int]=None,
                         seed:Optional[int]=None,
                         tags:Optional[dict[str,numpy.ndarray]]=None,
                         idx_bottom:Optional[numpy.ndarray]=None)

*MinTrace Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
y_insample: Insample values of size (base, insample_size). Only used with “wls_var”, “mint_cov”, “mint_shrink”.
y_hat_insample: Insample forecast values of size (base, insample_size). Only used with “wls_var”, “mint_cov”, “mint_shrink”
sigmah: Estimated standard deviation of the conditional marginal distribution.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).

Returns:
self: object, fitted reconciler.*

​

OptimalCombination.predict

 OptimalCombination.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                             level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

OptimalCombination.fit_predict

 OptimalCombination.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                                 idx_bottom:numpy.ndarray=None,
                                 y_insample:Optional[numpy.ndarray]=None, 
                                 y_hat_insample:Optional[numpy.ndarray]=No
                                 ne, sigmah:Optional[numpy.ndarray]=None,
                                 level:Optional[list[int]]=None,
                                 intervals_method:Optional[str]=None,
                                 num_samples:Optional[int]=None,
                                 seed:Optional[int]=None, tags:Optional[di
                                 ct[str,numpy.ndarray]]=None)

*MinTrace Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: Insample values of size (base, insample_size). Only used by wls_var, mint_cov, mint_shrink
y_hat_insample: Insample fitted values of size (base, insample_size). Only used by wls_var, mint_cov, mint_shrink
sigmah: Estimated standard deviation of the conditional marginal distribution.
level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.

Returns:
y_tilde: Reconciliated y_hat using the MinTrace approach.*

​

OptimalCombination.sample

 OptimalCombination.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

​

6. Emp. Risk Minimization

​

ERM

 ERM (method:str, lambda_reg:float=0.01)

*Optimal Combination Reconciliation Class.

The Empirical Risk Minimization reconciliation strategy relaxes the unbiasedness assumptions from previous reconciliation methods like MinT and optimizes square errors between the reconciled predictions and the validation data to obtain an optimal reconciliation matrix P.

The exact solution for $\mathbf{P}$ (method='closed') follows the expression: $\mathbf{P}^{*} = \left(\mathbf{S}^{\intercal}\mathbf{S}\right)^{-1}\mathbf{Y}^{\intercal}\hat{\mathbf{Y}}\left(\hat{\mathbf{Y}}\hat{\mathbf{Y}}\right)^{-1}$

The alternative Lasso regularized $\mathbf{P}$ solution (method='reg_bu') is useful when the observations of validation data is limited or the exact solution has low numerical stability. $\mathbf{P}^{*} = \text{argmin}_{\mathbf{P}} ||\mathbf{Y}-\mathbf{S} \mathbf{P} \hat{Y} ||^{2}_{2} + \lambda ||\mathbf{P}-\mathbf{P}_{\text{BU}}||_{1}$

Parameters:
method: str, one of closed, reg and reg_bu.
lambda_reg: float, l1 regularizer for reg and reg_bu.

References:
- Ben Taieb, S., & Koo, B. (2019). Regularized regression for hierarchical forecasting without unbiasedness conditions. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining KDD ’19 (p. 1337-1347). New York, NY, USA: Association for Computing Machinery..
*

​

ERM.fit

 ERM.fit (S, y_hat, y_insample, y_hat_insample,
          sigmah:Optional[numpy.ndarray]=None,
          intervals_method:Optional[str]=None,
          num_samples:Optional[int]=None, seed:Optional[int]=None,
          tags:Optional[dict[str,numpy.ndarray]]=None,
          idx_bottom:Optional[numpy.ndarray]=None)

*ERM Fit Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
y_insample: Train values of size (base, insample_size).
y_hat_insample: Insample train predictions of size (base, insample_size).
sigmah: Estimated standard deviation of the conditional marginal distribution.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).

Returns:
self: object, fitted reconciler.*

​

ERM.predict

 ERM.predict (S:numpy.ndarray, y_hat:numpy.ndarray,
              level:Optional[list[int]]=None)

*Predict using reconciler.

Predict using fitted mean and probabilistic reconcilers.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
level: float list 0-100, confidence levels for prediction intervals.

Returns:
y_tilde: Reconciliated predictions.*

​

ERM.fit_predict

 ERM.fit_predict (S:numpy.ndarray, y_hat:numpy.ndarray,
                  idx_bottom:numpy.ndarray=None,
                  y_insample:Optional[numpy.ndarray]=None,
                  y_hat_insample:Optional[numpy.ndarray]=None,
                  sigmah:Optional[numpy.ndarray]=None,
                  level:Optional[list[int]]=None,
                  intervals_method:Optional[str]=None,
                  num_samples:Optional[int]=None, seed:Optional[int]=None,
                  tags:Optional[dict[str,numpy.ndarray]]=None)

*ERM Reconciliation Method.

Parameters:
S: Summing matrix of size (base, bottom).
y_hat: Forecast values of size (base, horizon).
idx_bottom: Indices corresponding to the bottom level of S, size (bottom).
y_insample: Train values of size (base, insample_size).
y_hat_insample: Insample train predictions of size (base, insample_size).
sigmah: Estimated standard deviation of the conditional marginal distribution.
level: float list 0-100, confidence levels for prediction intervals.
intervals_method: Sampler for prediction intervals, one of normality, bootstrap, permbu.
num_samples: Number of samples for probabilistic coherent distribution.
seed: Seed for reproducibility.
tags: Each key is a level and each value its S indices.

Returns:
y_tilde: Reconciliated y_hat using the ERM approach.*

​

ERM.sample

 ERM.sample (num_samples:int)

*Sample probabilistic coherent distribution.

Generates n samples from a probabilistic coherent distribution. The method uses fitted mean and probabilistic reconcilers, defined by the intervals_method selected during the reconciler’s instantiation. Currently available: normality, bootstrap, permbu.

Parameters:
num_samples: int, number of samples generated from coherent distribution.

Returns:
samples: Coherent samples of size (num_series, horizon, num_samples).*

S @ y_hat_bottom_insample

​

References

​

General Reconciliation

• Orcutt, G.H., Watts, H.W., & Edwards, J.B.(1968). Data aggregation and information loss. The American Economic Review, 58 , 773(787).
  
• Disaggregation methods to expedite product line forecasting. Journal of Forecasting, 9 , 233–254. doi:10.1002/for.3980090304.
  
• An investigation of aggregate variable time series forecast strategies with specific subaggregate time series statistical correlation. Computers and Operations Research, 26 , 1133–1149. doi:10.1016/S0305-0548(99)00017-9.
  
• Hyndman, R.J., & Athanasopoulos, G. (2021). “Forecasting: principles and practice, 3rd edition: Chapter 11: Forecasting hierarchical and grouped series.”. OTexts: Melbourne, Australia. OTexts.com/fpp3 Accessed on July 2022.

​

Optimal Reconciliation

• Rob J. Hyndman, Roman A. Ahmed, George Athanasopoulos, Han Lin Shang. “Optimal Combination Forecasts for Hierarchical Time Series” (2010).
  
• Shanika L. Wickramasuriya, George Athanasopoulos and Rob J. Hyndman. “Optimal Combination Forecasts for Hierarchical Time Series” (2010).
  
• Ben Taieb, S., & Koo, B. (2019). Regularized regression for hierarchical forecasting without unbiasedness conditions. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining KDD ’19 (p. 1337-1347). New York, NY, USA: Association for Computing Machinery.
  

​

Hierarchical Probabilistic Coherent Predictions

• Puwasala Gamakumara Ph. D. dissertation. Monash University, Econometrics and Business Statistics. “Probabilistic Forecast Reconciliation”.
  
• Taieb, Souhaib Ben and Taylor, James W and Hyndman, Rob J. (2017). Coherent probabilistic forecasts for hierarchical time series. International conference on machine learning ICML.