Exogenous Variables

Exogenous variables can provide additional information to greatly improve forecasting accuracy. Some examples include price or future promotions variables for demand forecasting, and weather data for electricity load forecast. In this notebook we show an example on how to add different types of exogenous variables to NeuralForecast models for making day-ahead hourly electricity price forecasts (EPF) for France and Belgium markets.

All NeuralForecast models are capable of incorporating exogenous variables to model the following conditional predictive distribution: $\mathbb{P}(\mathbf{y}_{t+1:t+H} \;|\; \mathbf{y}_{[:t]},\; \mathbf{x}^{(h)}_{[:t]},\; \mathbf{x}^{(f)}_{[:t+H]},\; \mathbf{x}^{(s)} )$

where the regressors are static exogenous $\mathbf{x}^{(s)}$, historic exogenous $\mathbf{x}^{(h)}_{[:t]}$, exogenous available at the time of the prediction $\mathbf{x}^{(f)}_{[:t+H]}$ and autorregresive features $\mathbf{y}_{[:t]}$. Depending on the train loss, the model outputs can be point forecasts (location estimators) or uncertainty intervals (quantiles).

We will show you how to include exogenous variables in the data, specify variables to a model, and produce forecasts using future exogenous variables.

Important

This Guide assumes basic knowledge on the NeuralForecast library. For a minimal example visit the Getting Started guide.

You can run these experiments using GPU with Google Colab.

​

1. Libraries

!pip install neuralforecast

​

2. Load data

The df dataframe contains the target and exogenous variables past information to train the model. The unique_id column identifies the markets, ds contains the datestamps, and y the electricity price.

Include both historic and future temporal variables as columns. In this example, we are adding the system load (system_load) as historic data. For future variables, we include a forecast of how much electricity will be produced (gen_forecast) and day of week (week_day). Both the electricity system demand and offer impact the price significantly, including these variables to the model greatly improve performance, as we demonstrate in Olivares et al. (2022).

The distinction between historic and future variables will be made later as parameters of the model.

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('https://datasets-nixtla.s3.amazonaws.com/EPF_FR_BE.csv')
df['ds'] = pd.to_datetime(df['ds'])
df.head()

Tip

Calendar variables such as day of week, month, and year are very useful to capture long seasonalities.

plt.figure(figsize=(15,5))
plt.plot(df[df['unique_id']=='FR']['ds'], df[df['unique_id']=='FR']['y'])
plt.xlabel('Date')
plt.ylabel('Price [EUR/MWh]')
plt.grid()

Add the static variables in a separate static_df dataframe. In this example, we are using one-hot encoding of the electricity market. The static_df must include one observation (row) for each unique_id of the df dataframe, with the different statics variables as columns.

static_df = pd.read_csv('https://datasets-nixtla.s3.amazonaws.com/EPF_FR_BE_static.csv')
static_df.head()

​

3. Training with exogenous variables

We distinguish the exogenous variables by whether they reflect static or time-dependent aspects of the modeled data.

• Static exogenous variables: The static exogenous variables carry time-invariant information for each time series. When the model is built with global parameters to forecast multiple time series, these variables allow sharing information within groups of time series with similar static variable levels. Examples of static variables include designators such as identifiers of regions, groups of products, etc.

• Historic exogenous variables: This time-dependent exogenous variable is restricted to past observed values. Its predictive power depends on Granger-causality, as its past values can provide significant information about future values of the target variable $\mathbf{y}$.

• Future exogenous variables: In contrast with historic exogenous variables, future values are available at the time of the prediction. Examples include calendar variables, weather forecasts, and known events that can cause large spikes and dips such as scheduled promotions.

To add exogenous variables to the model, first specify the name of each variable from the previous dataframes to the corresponding model hyperparameter during initialization: futr_exog_list, hist_exog_list, and stat_exog_list. We also set horizon as 24 to produce the next day hourly forecasts, and set input_size to use the last 5 days of data as input.

from neuralforecast.auto import NHITS, BiTCN
from neuralforecast.core import NeuralForecast

import logging
logging.getLogger("pytorch_lightning").setLevel(logging.WARNING)

horizon = 24 # day-ahead daily forecast
models = [NHITS(h = horizon,
                input_size = 5*horizon,
                futr_exog_list = ['gen_forecast', 'week_day'], # <- Future exogenous variables
                hist_exog_list = ['system_load'], # <- Historical exogenous variables
                stat_exog_list = ['market_0', 'market_1'], # <- Static exogenous variables
                scaler_type = 'robust'),
          BiTCN(h = horizon,
                input_size = 5*horizon,
                futr_exog_list = ['gen_forecast', 'week_day'], # <- Future exogenous variables
                hist_exog_list = ['system_load'], # <- Historical exogenous variables
                stat_exog_list = ['market_0', 'market_1'], # <- Static exogenous variables
                scaler_type = 'robust',
                ),                
                ]

c:\Users\ospra\miniconda3\envs\neuralforecast\lib\site-packages\pytorch_lightning\utilities\parsing.py:199: Attribute 'loss' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['loss'])`.
Seed set to 1
Seed set to 1

Tip

When including exogenous variables always use a scaler by setting the scaler_type hyperparameter. The scaler will scale all the temporal features: the target variable y, historic and future variables.

Important

Make sure future and historic variables are correctly placed. Defining historic variables as future variables will lead to data leakage.

Next, pass the datasets to the df and static_df inputs of the fit method.

nf = NeuralForecast(models=models, freq='H')
nf.fit(df=df,
       static_df=static_df)

​

4. Forecasting with exogenous variables

Before predicting the prices, we need to gather the future exogenous variables for the day we want to forecast. Define a new dataframe (futr_df) with the unique_id, ds, and future exogenous variables. There is no need to add the target variable y and historic variables as they won’t be used by the model.

futr_df = pd.read_csv('https://datasets-nixtla.s3.amazonaws.com/EPF_FR_BE_futr.csv')
futr_df['ds'] = pd.to_datetime(futr_df['ds'])
futr_df.head()

Important

Make sure futr_df has informations for the entire forecast horizon. In this example, we are forecasting 24 hours ahead, so futr_df must have 24 rows for each time series.

Finally, use the predict method to forecast the day-ahead prices.

Y_hat_df = nf.predict(futr_df=futr_df)
Y_hat_df.head()

c:\Users\ospra\miniconda3\envs\neuralforecast\lib\site-packages\utilsforecast\processing.py:352: FutureWarning: 'H' is deprecated and will be removed in a future version, please use 'h' instead.
  freq = pd.tseries.frequencies.to_offset(freq)
c:\Users\ospra\miniconda3\envs\neuralforecast\lib\site-packages\utilsforecast\processing.py:404: FutureWarning: 'H' is deprecated and will be removed in a future version, please use 'h' instead.
  freq = pd.tseries.frequencies.to_offset(freq)
c:\Users\ospra\OneDrive\Phd\Repositories\neuralforecast\neuralforecast\tsdataset.py:91: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  self.temporal = torch.tensor(temporal, dtype=torch.float)
c:\Users\ospra\OneDrive\Phd\Repositories\neuralforecast\neuralforecast\tsdataset.py:95: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  self.static = torch.tensor(static, dtype=torch.float)
c:\Users\ospra\miniconda3\envs\neuralforecast\lib\site-packages\pytorch_lightning\trainer\connectors\data_connector.py:441: The 'predict_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=19` in the `DataLoader` to improve performance.
c:\Users\ospra\OneDrive\Phd\Repositories\neuralforecast\neuralforecast\core.py:179: FutureWarning: In a future version the predictions will have the id as a column. You can set the `NIXTLA_ID_AS_COL` environment variable to adopt the new behavior and to suppress this warning.
  warnings.warn(

Predicting: |          | 0/? [00:00<?, ?it/s]

Predicting: |          | 0/? [00:00<?, ?it/s]

import matplotlib.pyplot as plt

plot_df = df[df['unique_id']=='FR'].tail(24*5).reset_index(drop=True)
Y_hat_df = Y_hat_df.reset_index(drop=False)
Y_hat_df = Y_hat_df[Y_hat_df['unique_id']=='FR']

plot_df = pd.concat([plot_df, Y_hat_df ]).set_index('ds') # Concatenate the train and forecast dataframes

plot_df[['y', 'NHITS', 'BiTCN']].plot(linewidth=2)
plt.axvline('2016-11-01', color='red')
plt.ylabel('Price [EUR/MWh]', fontsize=12)
plt.xlabel('Date', fontsize=12)
plt.grid()

In summary, to add exogenous variables to a model make sure to follow the next steps:

1. Add temporal exogenous variables as columns to the main dataframe (df).
2. Add static exogenous variables with the static_df dataframe.
3. Specify the name for each variable in the corresponding model hyperparameter.
4. If the model uses future exogenous variables, pass the future dataframe (futr_df) to the predict method.

​

References

• Kin G. Olivares, Cristian Challu, Grzegorz Marcjasz, Rafał Weron, Artur Dubrawski, Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx, International Journal of Forecasting

• Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler-Canseco, Artur Dubrawski (2021). NHITS: Neural Hierarchical Interpolation for Time Series Forecasting. Accepted at AAAI 2023.