Skip to main content
On top of continuous numerical features, NeuralForecast models also support categorical features. These features take discrete values that may change over time (e.g., day of the week, month of the year, etc.). In this tutorial, we explore how categorical features are handled in NeuralForecast and how you can incorporate them when training a model.
Important Not all models support exogenous features, either numerical or categorical. Refer to the model list to see which model supports what type of exogenous features.

​
How it works

NeuralForecast handles categorical features with learned embeddings. Internally, it: - builds a panel-wide vocabulary at fit time. This creates a mapping from category to index for each feature across all series. Index 0 is reserved for unseen categories during training. - routes the categorical features around the scalers. Categorical features are not scaled, unlike numerical features. - learns an embedding for each feature. One vector per category is created and is learned jointly with other continuous feaetures. - concatenates each feature’s embeddings with the continous features and feeds the result to the model. By using learned embeddings, models learn a rich representation of the features which usually results in better forecasts than using a traditional label encoder. Note that the current handling of static exogenous features stays unchanged; i.e., they are not embedded.

​
Choosing the embdding dimension

Choosing the right embedding dimension determines the representational capacity of the model. There are four strategies available. Note that the cardinality is the number of unique categories for a feature.
StrategyFormulaCharacter
"fastai" (default)min(50, ceil(1.6 · cardinality^0.56))Balanced, cardinality-aware, sublinear growth
"sqrt"min(50, ceil(sqrt(cardinality)))Compact; smallest for mid/high cardinality
"half"min(50, (cardinality + 1) // 2)Generous for low cardinality; saturates fast
an integer (e.g. 8)exactly that valueUniform across all features
Be aware of the tradeoff when choosing the emebdding’s dimension. A larger dimension means a larger representational capacity, but it also means more parameters to train and a risk of overfitting.
  • “fastai” (default). It’s a well-established rule of thumb that scales nicely to large vocabularies, and rarely needs tuning.
  • “sqrt”. Use for high-cardinality features or limited data. It produces the most compact embeddings for mid-to-high cardinality, which means fewer parameters and stronger regularization.
  • “half”. Use when categories are few but highly informative and you have enough data to fit them. Tt gives low-cardinality features more capacity. Note it saturates at the cap quickly, so for high cardinality it behaves like a flat 50.
  • An explicit integer. Use when you want uniform, predictable sizes, want to enforce a specific dimension, or want to tune cat_emb_dim as a hyperparameter.

​
What is currently supported

This is a large change that is being implemented in multiple phases. Currently, categorical features are support for: - univariate models - non-recurrent models

​
Upcoming work

All elements listed here do not support the use of categorical features: - recurrent models - multivariate models - explainability with categorical features - simulation with categorical features

​
Example

Let’s see a simple example of how categorical features can be incorporated in NeuralForecast 
import pandas as pd
from neuralforecast import NeuralForecast
from neuralforecast.models import NHITS
from neuralforecast.utils import AirPassengersPanel

df = AirPassengersPanel[["unique_id", "ds", "y"]].copy()
df["ds"] = pd.to_datetime(df["ds"])
Now, let’s create a categorical feature called “month”. This is a known feature in the future. 
df["month"] = df["ds"].dt.month

test = df.groupby("unique_id").tail(12)
train = df.drop(test.index)
To include categorical features, we must specify four hyperparameters: 1. futr_exog_list or hist_exog_list. These parameters take a list of features, depending on whether they are historical (we only know their past values), or future (we know their future values). They can contain both numerical and categorical features. 2. cat_exog_list. This parameter specifies which features are categorical. That way, we automatically resolve which features are categorical from futr_exog_list and hist_exog_list. 3. categorical_cardinalities. A dictionary that assigns the cardinality (the number of unique values) for each categorical feature. 4. cat_embed_dim. The strategy to set the dimension of the learned embeddings. In our current example, “month” is a categorical feature with known future values, so it is included in futr_exog_list. Since there are 12 months in a year, its cardinality is 12. Below is an example of traning a NHITS model with a categorical feature. 
model = NHITS(
    h=12, 
    input_size=24, 
    max_steps=500,
    futr_exog_list=["month"],
    cat_exog_list=["month"],
    categorical_cardinalities={"month": 12},
    cat_emb_dim="fastai",
)
nf = NeuralForecast(models=[model], freq="ME")
nf.fit(train)
Once trained, the model can be used for inference. Since we specified “month” as a future exogenous feature, we must provide its values over the forecast horizon. 
futr = test[["unique_id", "ds", "month"]]
preds = nf.predict(futr_df=futr)