> ## Documentation Index
> Fetch the complete documentation index at: https://nixtlaverse.nixtla.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Categorical features

On top of continuous numerical features, NeuralForecast models also
support categorical features. These features take discrete values that
may change over time (e.g., day of the week, month of the year, etc.).

In this tutorial, we explore how categorical features are handled in
NeuralForecast and how you can incorporate them when training a model.

> **Important**
>
> Not all models support exogenous features, either numerical or
> categorical. Refer to the [model list](./overview.html) to see which
> model supports what type of exogenous features.

## How it works

NeuralForecast handles categorical features with learned embeddings.
Internally, it: - builds a panel-wide vocabulary at `fit` time. This
creates a mapping from category to index for each feature across all
series. Index 0 is reserved for unseen categories during training. -
routes the categorical features around the scalers. Categorical features
are not scaled, unlike numerical features. - learns an embedding for
each feature. One vector per category is created and is learned jointly
with other continuous feaetures. - concatenates each feature’s
embeddings with the continous features and feeds the result to the
model.

By using learned embeddings, models learn a rich representation of the
features which usually results in better forecasts than using a
traditional label encoder.

Note that the current handling of static exogenous features stays
unchanged; i.e., they are not embedded.

### Choosing the embdding dimension

Choosing the right embedding dimension determines the representational
capacity of the model.

There are four strategies available. Note that the cardinality is the
number of unique categories for a feature.

| Strategy              | Formula                                 | Character                                     |
| --------------------- | --------------------------------------- | --------------------------------------------- |
| `"fastai"` (default)  | `min(50, ceil(1.6 · cardinality^0.56))` | Balanced, cardinality-aware, sublinear growth |
| `"sqrt"`              | `min(50, ceil(sqrt(cardinality)))`      | Compact; smallest for mid/high cardinality    |
| `"half"`              | `min(50, (cardinality + 1) // 2)`       | Generous for low cardinality; saturates fast  |
| an integer (e.g. `8`) | exactly that value                      | Uniform across all features                   |

Be aware of the tradeoff when choosing the emebdding’s dimension. A
larger dimension means a larger representational capacity, but it also
means more parameters to train and a risk of overfitting.

* “fastai” (default). It’s a well-established rule of thumb that
  scales nicely to large vocabularies, and rarely needs tuning.
* “sqrt”. Use for high-cardinality features or limited data. It
  produces the most compact embeddings for mid-to-high cardinality,
  which means fewer parameters and stronger regularization.
* “half”. Use when categories are few but highly informative and you
  have enough data to fit them. Tt gives low-cardinality features more
  capacity. Note it saturates at the cap quickly, so for high
  cardinality it behaves like a flat 50.
* An explicit integer. Use when you want uniform, predictable sizes,
  want to enforce a specific dimension, or want to tune cat\_emb\_dim as
  a hyperparameter.

### What is currently supported

This is a large change that is being implemented in multiple phases.
Currently, categorical features are support for: - univariate models -
non-recurrent models

### Upcoming work

All elements listed here do not support the use of categorical
features: - recurrent models - multivariate models - explainability with
categorical features - simulation with categorical features

## Example

Let’s see a simple example of how categorical features can be
incorporated in NeuralForecast

```python theme={null}
import pandas as pd
from neuralforecast import NeuralForecast
from neuralforecast.models import NHITS
from neuralforecast.utils import AirPassengersPanel
```

```python theme={null}
df = AirPassengersPanel[["unique_id", "ds", "y"]].copy()
df["ds"] = pd.to_datetime(df["ds"])
```

Now, let’s create a categorical feature called “month”. This is a known
feature in the future.

```python theme={null}
df["month"] = df["ds"].dt.month
```

```python theme={null}
test = df.groupby("unique_id").tail(12)
train = df.drop(test.index)
```

To include categorical features, we must specify four
hyperparameters: 1. `futr_exog_list` or `hist_exog_list`. These
parameters take a list of features, depending on whether they are
historical (we only know their past values), or future (we know their
future values). They can contain both numerical and categorical
features. 2. `cat_exog_list`. This parameter specifies which features
are categorical. That way, we automatically resolve which features are
categorical from `futr_exog_list` and `hist_exog_list`. 3.
`categorical_cardinalities`. A dictionary that assigns the cardinality
(the number of unique values) for each categorical feature. 4.
`cat_embed_dim`. The strategy to set the dimension of the learned
embeddings.

In our current example, “month” is a categorical feature with known
future values, so it is included in `futr_exog_list`. Since there are 12
months in a year, its cardinality is 12.

Below is an example of traning a NHITS model with a categorical feature.

```python theme={null}
model = NHITS(
    h=12, 
    input_size=24, 
    max_steps=500,
    futr_exog_list=["month"],
    cat_exog_list=["month"],
    categorical_cardinalities={"month": 12},
    cat_emb_dim="fastai",
)
nf = NeuralForecast(models=[model], freq="ME")
nf.fit(train)
```

Once trained, the model can be used for inference. Since we specified
“month” as a future exogenous feature, we must provide its values over
the forecast horizon.

```python theme={null}
futr = test[["unique_id", "ds", "month"]]
preds = nf.predict(futr_df=futr)
```