> ## Documentation Index
> Fetch the complete documentation index at: https://nixtlaverse.nixtla.io/llms.txt
> Use this file to discover all available pages before exploring further.

# TimesNet

> TimesNet: 2D-variation modeling with Inception blocks for capturing intraperiod and interperiod temporal patterns in univariate time series forecasting.

The TimesNet univariate model tackles the challenge of modeling multiple
intraperiod and interperiod temporal variations.

The architecture has the following distinctive features: - An embedding
layer that maps the input sequence into a latent space. - Transformation
of 1D time series into 2D tensors, based on periods found by FFT. - A
convolutional Inception block that captures temporal variations at
different scales and between periods.

**References**

* [Haixu Wu and Tengge Hu and Yong Liu and Hang Zhou
  and Jianmin Wang and Mingsheng Long. TimesNet: Temporal 2D-Variation
  Modeling for General Time Series
  Analysis](https://openreview.net/pdf?id=ju_Uqw384Oq) - Based on the
  implementation in [https://github.com/thuml/Time-Series-Library](https://github.com/thuml/Time-Series-Library) (license:
  [https://github.com/thuml/Time-Series-Library/blob/main/LICENSE](https://github.com/thuml/Time-Series-Library/blob/main/LICENSE))

![Figure 1. TimesNet Architecture.](https://mintcdn.com/nixtla/wOkzptAA8LlzXeB0/neuralforecast/imgs_models/timesnet.png?fit=max\&auto=format\&n=wOkzptAA8LlzXeB0\&q=85\&s=c74c9fc10cc106efa6f49866c20ec126)

*Figure 1. TimesNet
Architecture.*

## 1. TimesNet

### `TimesNet`

```python theme={null}
TimesNet(
    h,
    input_size,
    stat_exog_list=None,
    hist_exog_list=None,
    futr_exog_list=None,
    cat_exog_list=None,
    categorical_cardinalities=None,
    cat_emb_dim="fastai",
    exclude_insample_y=False,
    hidden_size=64,
    dropout=0.1,
    conv_hidden_size=64,
    top_k=5,
    num_kernels=6,
    encoder_layers=2,
    loss=MAE(),
    valid_loss=None,
    max_steps=1000,
    learning_rate=0.0001,
    num_lr_decays=-1,
    early_stop_patience_steps=-1,
    val_monitor="ptl/val_loss",
    val_check_steps=100,
    batch_size=32,
    valid_batch_size=None,
    windows_batch_size=64,
    inference_windows_batch_size=256,
    start_padding_enabled=False,
    training_data_availability_threshold=0.0,
    step_size=1,
    scaler_type="standard",
    random_seed=1,
    drop_last_loader=False,
    alias=None,
    optimizer=None,
    optimizer_kwargs=None,
    lr_scheduler=None,
    lr_scheduler_kwargs=None,
    dataloader_kwargs=None,
    **trainer_kwargs
)
```

Bases: `BaseModel`

TimesNet

The TimesNet univariate model tackles the challenge of modeling multiple intraperiod and interperiod temporal variations.

**Parameters:**

| Name | Type | Description | Default |
| - | - | - | - |
| `h` | `int` | Forecast horizon. | *required* |
| `input_size` | `int` | Length of input window (lags). | *required* |
| `stat_exog_list` | `list of str` | optional (default=None), Static exogenous columns. | `None` |
| `hist_exog_list` | `list of str` | optional (default=None), Historic exogenous columns. | `None` |
| `futr_exog_list` | `list of str` | optional (default=None), Future exogenous columns. | `None` |
| `cat_exog_list` | `list of str` | optional (default=None), exogenous columns (from `hist_exog_list` / `futr_exog_list`) to embed instead of scale. | `None` |
| `categorical_cardinalities` | `dict` | optional (default=None), mapping from each categorical column to its number of distinct categories. | `None` |
| `cat_emb_dim` | `str or int` | categorical embedding size strategy ('fastai', 'sqrt', 'half') or an explicit integer. | `'fastai'` |
| `exclude_insample_y` | `bool` | The model skips the autoregressive features y\[t-input\_size:t] if True. | `False` |
| `hidden_size` | `int` | Size of embedding for embedding and encoders. | `64` |
| `dropout` | `float` | Dropout for embeddings. | `0.1` |
| `conv_hidden_size` | `int` | Channels of the Inception block. | `64` |
| `top_k` | `int` | Number of periods. | `5` |
| `num_kernels` | `int` | Number of kernels for the Inception block. | `6` |
| `encoder_layers` | `int` | Number of encoder layers. | `2` |
| `loss` | `PyTorch module` | Instantiated train loss class from [losses collection](https://nixtlaverse.nixtla.io/neuralforecast/losses.pytorch.html.md). | `MAE()` |
| `valid_loss` | `PyTorch module` | Instantiated validation loss class from [losses collection](https://nixtlaverse.nixtla.io/neuralforecast/losses.pytorch.html.md). | `None` |
| `max_steps` | `int` | Maximum number of training steps. | `1000` |
| `learning_rate` | `float` | Learning rate. | `0.0001` |
| `num_lr_decays` | `int` | Number of learning rate decays, evenly distributed across max\_steps. If -1, no learning rate decay is performed. | `-1` |
| `early_stop_patience_steps` | `int` | Number of validation iterations before early stopping. If -1, no early stopping is performed. | `-1` |
| `val_monitor` | `str` | metric to monitor for early stopping. Valid options: "ptl/val\_loss", "valid\_loss", "train\_loss". Default: "ptl/val\_loss". | `'ptl/val_loss'` |
| `val_check_steps` | `int` | Number of training steps between every validation loss check. | `100` |
| `batch_size` | `int` | Number of different series in each batch. | `32` |
| `valid_batch_size` | `int` | Number of different series in each validation and test batch, if None uses batch\_size. | `None` |
| `windows_batch_size` | `int` | Number of windows to sample in each training batch. | `64` |
| `inference_windows_batch_size` | `int` | Number of windows to sample in each inference batch. | `256` |
| `start_padding_enabled` | `bool` | If True, the model will pad the time series with zeros at the beginning by input size. | `False` |
| `training_data_availability_threshold` | `Union[float, List[float]]` | minimum fraction of valid data points required for training windows. Single float applies to both insample and outsample; list of two floats specifies \[insample\_fraction, outsample\_fraction]. Default 0.0 allows windows with only 1 valid data point (current behavior). | `0.0` |
| `step_size` | `int` | Step size between each window of temporal data. | `1` |
| `scaler_type` | `str` | Type of scaler for temporal inputs normalization see [temporal scalers](https://github.com/Nixtla/neuralforecast/blob/main/neuralforecast/common/_scalers.py). | `'standard'` |
| `random_seed` | `int` | Random\_seed for pytorch initializer and numpy generators. | `1` |
| `drop_last_loader` | `bool` | If True `TimeSeriesDataLoader` drops last non-full batch. | `False` |
| `alias` | `str` | optional (default=None), Custom name of the model. | `None` |
| `optimizer` | `Subclass of 'torch.optim.Optimizer'` | optional (default=None), User specified optimizer instead of the default choice (Adam). | `None` |
| `optimizer_kwargs` | `dict` | optional (default=None), List of parameters used by the user specified `optimizer`. | `None` |
| `lr_scheduler` | `Subclass of 'torch.optim.lr_scheduler.LRScheduler'` | optional, user specified lr\_scheduler instead of the default choice (StepLR). | `None` |
| `lr_scheduler_kwargs` | `dict` | optional, list of parameters used by the user specified `lr_scheduler`. | `None` |
| `dataloader_kwargs` | `dict` | optional (default=None), List of parameters passed into the PyTorch Lightning dataloader by the `TimeSeriesDataLoader`. | `None` |
| `**trainer_kwargs` | `int` | keyword trainer arguments inherited from [PyTorch Lightning's trainer](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer). | |

<details open markdown="1">
  <summary>References</summary>

  * [Haixu Wu and Tengge Hu and Yong Liu and Hang Zhou and Jianmin Wang and Mingsheng Long. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. https://openreview.net/pdf?id=ju\_Uqw384Oq](https://openreview.net/pdf?id=ju_Uqw384Oq)
</details>

#### `TimesNet.fit`

```python theme={null}
fit(
    dataset, val_size=0, test_size=0, random_seed=None, distributed_config=None
)
```

Fit.

The `fit` method, optimizes the neural network's weights using the
initialization parameters (`learning_rate`, `windows_batch_size`, ...)
and the `loss` function as defined during the initialization.
Within `fit` we use a PyTorch Lightning `Trainer` that
inherits the initialization's `self.trainer_kwargs`, to customize
its inputs, see [PL's trainer arguments](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer).

The method is designed to be compatible with SKLearn-like classes
and in particular to be compatible with the StatsForecast library.

By default the `model` is not saving training checkpoints to protect
disk memory, to get them change `enable_checkpointing=True` in `__init__`.

**Parameters:**

| Name | Type | Description | Default |
| - | - | - | - |
| `dataset` | `TimeSeriesDataset` | NeuralForecast's `TimeSeriesDataset`, see [documentation](https://nixtlaverse.nixtla.io/neuralforecast/tsdataset.html.md). | *required* |
| `val_size` | `int` | Validation size for temporal cross-validation. | `0` |
| `random_seed` | `int` | Random seed for pytorch initializer and numpy generators, overwrites model.**init**'s. | `None` |
| `test_size` | `int` | Test size for temporal cross-validation. | `0` |

**Returns:**

| Type | Description |
| - | - |
| None | |

#### `TimesNet.predict`

```python theme={null}
predict(
    dataset,
    test_size=None,
    step_size=1,
    random_seed=None,
    quantiles=None,
    h=None,
    explainer_config=None,
    **data_module_kwargs
)
```

Predict.

Neural network prediction with PL's `Trainer` execution of `predict_step`.

**Parameters:**

| Name | Type | Description | Default |
| - | - | - | - |
| `dataset` | `TimeSeriesDataset` | NeuralForecast's `TimeSeriesDataset`, see [documentation](https://nixtlaverse.nixtla.io/neuralforecast/tsdataset.html.md). | *required* |
| `test_size` | `int` | Test size for temporal cross-validation. | `None` |
| `step_size` | `int` | Step size between each window. | `1` |
| `random_seed` | `int` | Random seed for pytorch initializer and numpy generators, overwrites model.**init**'s. | `None` |
| `quantiles` | `list` | Target quantiles to predict. | `None` |
| `h` | `int` | Prediction horizon, if None, uses the model's fitted horizon. Defaults to None. | `None` |
| `explainer_config` | `dict` | configuration for explanations. | `None` |
| `**data_module_kwargs` | `dict` | PL's TimeSeriesDataModule args, see [documentation](https://pytorch-lightning.readthedocs.io/en/1.6.1/extensions/datamodules.html#using-a-datamodule). | |

**Returns:**

| Type | Description |
| - | - |
| None | |

### Usage Example

```python theme={null}
import pandas as pd
import matplotlib.pyplot as plt

from neuralforecast import NeuralForecast
from neuralforecast.losses.pytorch import DistributionLoss
from neuralforecast.utils import AirPassengersPanel, AirPassengersStatic

Y_train_df = AirPassengersPanel[AirPassengersPanel.ds<AirPassengersPanel['ds'].values[-12]] # 132 train
Y_test_df = AirPassengersPanel[AirPassengersPanel.ds>=AirPassengersPanel['ds'].values[-12]].reset_index(drop=True) # 12 test

model = TimesNet(h=12,
                 input_size=24,
                 hidden_size = 16,
                 conv_hidden_size = 32,
                 loss=DistributionLoss(distribution='Normal', level=[80, 90]),
                 scaler_type='standard',
                 learning_rate=1e-3,
                 max_steps=100,
                 val_check_steps=50,
                 early_stop_patience_steps=2)

nf = NeuralForecast(
    models=[model],
    freq='ME'
)
nf.fit(df=Y_train_df, static_df=AirPassengersStatic, val_size=12)
forecasts = nf.predict(futr_df=Y_test_df)

Y_hat_df = forecasts.reset_index(drop=False).drop(columns=['unique_id','ds'])
plot_df = pd.concat([Y_test_df, Y_hat_df], axis=1)
plot_df = pd.concat([Y_train_df, plot_df])

if model.loss.is_distribution_output:
    plot_df = plot_df[plot_df.unique_id=='Airline1'].drop('unique_id', axis=1)
    plt.plot(plot_df['ds'], plot_df['y'], c='black', label='True')
    plt.plot(plot_df['ds'], plot_df['TimesNet-median'], c='blue', label='median')
    plt.fill_between(x=plot_df['ds'][-12:], 
                    y1=plot_df['TimesNet-lo-90'][-12:].values, 
                    y2=plot_df['TimesNet-hi-90'][-12:].values,
                    alpha=0.4, label='level 90')
    plt.grid()
    plt.legend()
    plt.plot()
else:
    plot_df = plot_df[plot_df.unique_id=='Airline1'].drop('unique_id', axis=1)
    plt.plot(plot_df['ds'], plot_df['y'], c='black', label='True')
    plt.plot(plot_df['ds'], plot_df['TimesNet'], c='blue', label='Forecast')
    plt.legend()
    plt.grid()
```

## 2. Auxiliary Functions

### `Inception_Block_V1`

```python theme={null}
Inception_Block_V1(in_channels, out_channels, num_kernels=6, init_weight=True)
```

Bases: `Module`

Inception\_Block\_V1

### `TimesBlock`

```python theme={null}
TimesBlock(input_size, h, k, hidden_size, conv_hidden_size, num_kernels)
```

Bases: `Module`

TimesBlock

### `FFT_for_Period`

```python theme={null}
FFT_for_Period(x, k=2)
```


## Related topics

- [Time Series Scaling](https://nixtlaverse.nixtla.io/neuralforecast/docs/capabilities/time_series_scaling.html.md)
- [Forecasting Models](https://nixtlaverse.nixtla.io/neuralforecast/docs/capabilities/overview.html.md)
- [Time-LLM](https://nixtlaverse.nixtla.io/neuralforecast/models.timellm.html.md)
- [Automatic Forecasting](https://nixtlaverse.nixtla.io/neuralforecast/models.html.md)
- [Temporal Fusion Transformer Forecasting in Python](https://nixtlaverse.nixtla.io/neuralforecast/models.tft.html.md)
