Geographical Hierarchical Forecasting on Australian Prison Population DataIn many applications, a set of time series is hierarchically organized. Examples include the presence of geographic levels, products, or categories that define different types of aggregations. In such scenarios, forecasters are often required to provide predictions for all disaggregate and aggregate series. A natural desire is for those predictions to be “coherent”, that is, for the bottom series to add up precisely to the forecasts of the aggregated series. In this notebook we present an example on how to use
HierarchicalForecast
to produce coherent forecasts between
geographical levels. We will use the Australian Prison Population
dataset.
We will first load the dataset and produce base forecasts using an ETS
model from StatsForecast
, and then reconciliate the forecasts with
several reconciliation algorithms from HierarchicalForecast
. Finally,
we show the performance is comparable with the results reported by the
Forecasting: Principles and
Practice which uses the R package
fable.
You can run these experiments using CPU or GPU with Google Colab.
Country | State | Gender | Legal | Indigenous | ds | y | |
---|---|---|---|---|---|---|---|
0 | Australia | ACT | Female | Remanded | ATSI | 2005-04-01 | 0 |
1 | Australia | ACT | Female | Remanded | Non-ATSI | 2005-04-01 | 2 |
2 | Australia | ACT | Female | Sentenced | ATSI | 2005-04-01 | 0 |
3 | Australia | ACT | Female | Sentenced | Non-ATSI | 2005-04-01 | 5 |
4 | Australia | ACT | Male | Remanded | ATSI | 2005-04-01 | 7 |
aggregate
function from HierarchicalForecast
we can get
the full set of time series.
unique_id | ds | y | |
---|---|---|---|
0 | Australia | 2005-04-01 | 24.296 |
1 | Australia | 2005-07-01 | 24.643 |
2 | Australia | 2005-10-01 | 24.511 |
3 | Australia | 2006-01-01 | 24.393 |
4 | Australia | 2006-04-01 | 24.524 |
unique_id | Australia/ACT/Female/Remanded | Australia/ACT/Female/Sentenced | Australia/ACT/Male/Remanded | Australia/ACT/Male/Sentenced | |
---|---|---|---|---|---|
0 | Australia | 1.0 | 1.0 | 1.0 | 1.0 |
1 | Australia/ACT | 1.0 | 1.0 | 1.0 | 1.0 |
2 | Australia/NSW | 0.0 | 0.0 | 0.0 | 0.0 |
3 | Australia/NT | 0.0 | 0.0 | 0.0 | 0.0 |
4 | Australia/QLD | 0.0 | 0.0 | 0.0 | 0.0 |
Y_df
using the ETS
model. Observe that Y_hat_df
contains the
forecasts but they are not coherent.
unique_id | ds | y | |
---|---|---|---|
40 | Australia | 2015-04-01 | 35.271 |
41 | Australia | 2015-07-01 | 35.921 |
42 | Australia | 2015-10-01 | 36.067 |
43 | Australia | 2016-01-01 | 36.983 |
44 | Australia | 2016-04-01 | 37.830 |
… | … | … | … |
2155 | Australia/WA/Male/Sentenced | 2016-01-01 | 3.894 |
2156 | Australia/WA/Male/Sentenced | 2016-04-01 | 3.876 |
2157 | Australia/WA/Male/Sentenced | 2016-07-01 | 3.969 |
2158 | Australia/WA/Male/Sentenced | 2016-10-01 | 4.076 |
2159 | Australia/WA/Male/Sentenced | 2017-01-01 | 4.088 |
unique_id | ds | y | |
---|---|---|---|
0 | Australia | 2005-04-01 | 24.296 |
1 | Australia | 2005-07-01 | 24.643 |
2 | Australia | 2005-10-01 | 24.511 |
3 | Australia | 2006-01-01 | 24.393 |
4 | Australia | 2006-04-01 | 24.524 |
… | … | … | … |
2147 | Australia/WA/Male/Sentenced | 2014-01-01 | 3.614 |
2148 | Australia/WA/Male/Sentenced | 2014-04-01 | 3.635 |
2149 | Australia/WA/Male/Sentenced | 2014-07-01 | 3.692 |
2150 | Australia/WA/Male/Sentenced | 2014-10-01 | 3.726 |
2151 | Australia/WA/Male/Sentenced | 2015-01-01 | 3.780 |
HierarchicalReconciliation
class. Since the hierarchy structure is not
strict, we can’t use methods such as TopDown
or MiddleOut
. In this
example we use BottomUp
and MinTrace
.
Y_rec_df
contains the reconciled forecasts.
unique_id | ds | AutoETS | AutoETS/BottomUp | AutoETS/MinTrace_method-mint_shrink | |
---|---|---|---|---|---|
0 | Australia | 2015-04-01 | 34.799497 | 34.946476 | 34.923548 |
1 | Australia | 2015-07-01 | 35.192638 | 35.410342 | 35.432421 |
2 | Australia | 2015-10-01 | 35.188216 | 35.580849 | 35.473386 |
3 | Australia | 2016-01-01 | 35.888628 | 35.951878 | 35.939526 |
4 | Australia | 2016-04-01 | 36.045437 | 36.416829 | 36.245158 |
HierarchicalForecast
package includes the HierarchicalEvaluation
class to evaluate the different hierarchies and also is capable of
compute scaled metrics compared to a benchmark model.
level | metric | Base | AutoETS/BottomUp | AutoETS/MinTrace_method-mint_shrink | |
---|---|---|---|---|---|
0 | Total | mase | 1.36 | 1.07 | 1.17 |
1 | State | mase | 1.53 | 1.55 | 1.59 |
2 | Legal status | mase | 2.40 | 2.48 | 2.38 |
3 | Gender | mase | 1.08 | 0.82 | 0.93 |
4 | Bottom | mase | 2.16 | 2.16 | 2.14 |
5 | Overall | mase | 1.99 | 1.98 | 1.98 |