> ## Documentation Index > Fetch the complete documentation index at: https://nixtlaverse.nixtla.io/llms.txt > Use this file to discover all available pages before exploring further. > TSMixer: MLP-based multivariate forecasting with time and feature mixing. Stacked mixing layers learn temporal and cross-sectional representations jointly. # TSMixer Time-Series Mixer (`TSMixer`) is a MLP-based multivariate time-series forecasting model. `TSMixer` jointly learns temporal and cross-sectional representations of the time-series by repeatedly combining time- and feature information using stacked mixing layers. A mixing layer consists of a sequential time- and feature Multi Layer Perceptron (`MLP`). Note: this model cannot handle exogenous inputs. If you want to use additional exogenous inputs, use `TSMixerx`. Figure 1. TSMixer for multivariate time series forecasting. *Figure 1. TSMixer for multivariate time series forecasting.* ## 1. TSMixer ### `TSMixer` ```python theme={null} TSMixer( h, input_size, n_series, futr_exog_list=None, hist_exog_list=None, stat_exog_list=None, exclude_insample_y=False, n_block=2, ff_dim=64, dropout=0.9, revin=True, loss=MAE(), valid_loss=None, max_steps=1000, learning_rate=0.001, num_lr_decays=-1, early_stop_patience_steps=-1, val_monitor="ptl/val_loss", val_check_steps=100, batch_size=32, valid_batch_size=None, windows_batch_size=32, inference_windows_batch_size=32, start_padding_enabled=False, training_data_availability_threshold=0.0, step_size=1, scaler_type="identity", random_seed=1, drop_last_loader=False, alias=None, optimizer=None, optimizer_kwargs=None, lr_scheduler=None, lr_scheduler_kwargs=None, dataloader_kwargs=None, **trainer_kwargs ) ``` Bases: [BaseModel](#neuralforecast.common._base_model.BaseModel) TSMixer Time-Series Mixer (`TSMixer`) is a MLP-based multivariate time-series forecasting model. `TSMixer` jointly learns temporal and cross-sectional representations of the time-series by repeatedly combining time- and feature information using stacked mixing layers. A mixing layer consists of a sequential time- and feature Multi Layer Perceptron (`MLP`). **Parameters:** | Name | Type | Description | Default | | -------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------- | | `h` | [int](#int) | forecast horizon. | *required* | | `input_size` | [int](#int) | considered autorregresive inputs (lags), y=\[1,2,3,4] input\_size=2 -> lags=\[1,2]. | *required* | | `n_series` | [int](#int) | number of time-series. | *required* | | `futr_exog_list` | str list | future exogenous columns. | None | | `hist_exog_list` | str list | historic exogenous columns. | None | | `stat_exog_list` | str list | static exogenous columns. | None | | `exclude_insample_y` | [bool](#bool) | if True excludes the target variable from the input features. | False | | `n_block` | [int](#int) | number of mixing layers in the model. | 2 | | `ff_dim` | [int](#int) | number of units for the second feed-forward layer in the feature MLP. | 64 | | `dropout` | [float](#float) | dropout rate between (0, 1) . | 0.9 | | `revin` | [bool](#bool) | if True uses Reverse Instance Normalization to process inputs and outputs. | True | | `loss` | PyTorch module | instantiated train loss class from [losses collection](./losses.pytorch.html). | [MAE](#neuralforecast.losses.pytorch.MAE)() | | `valid_loss` | PyTorch module | instantiated valid loss class from [losses collection](./losses.pytorch.html). | None | | `max_steps` | [int](#int) | maximum number of training steps. | 1000 | | `learning_rate` | [float](#float) | Learning rate between (0, 1). | 0.001 | | `num_lr_decays` | [int](#int) | Number of learning rate decays, evenly distributed across max\_steps. | -1 | | `early_stop_patience_steps` | [int](#int) | Number of validation iterations before early stopping. | -1 | | `val_monitor` | [str](#str) | metric to monitor for early stopping. Valid options: "ptl/val\_loss", "valid\_loss", "train\_loss". Default: "ptl/val\_loss". | 'ptl/val\_loss' | | `val_check_steps` | [int](#int) | Number of training steps between every validation loss check. | 100 | | `batch_size` | [int](#int) | number of different series in each batch. | 32 | | `valid_batch_size` | [int](#int) | number of different series in each validation and test batch, if None uses batch\_size. | None | | `windows_batch_size` | [int](#int) | number of windows to sample in each training batch, default uses all. | 32 | | `inference_windows_batch_size` | [int](#int) | number of windows to sample in each inference batch, -1 uses all. | 32 | | `start_padding_enabled` | [bool](#bool) | if True, the model will pad the time series with zeros at the beginning, by input size. | False | | `training_data_availability_threshold` | [Union](#Union)\[[float](#float), [List](#List)\[[float](#float)]] | minimum fraction of valid data points required for training windows. Single float applies to both insample and outsample; list of two floats specifies \[insample\_fraction, outsample\_fraction]. Default 0.0 allows windows with only 1 valid data point (current behavior). | 0.0 | | `step_size` | [int](#int) | step size between each window of temporal data. | 1 | | `scaler_type` | [str](#str) | type of scaler for temporal inputs normalization see [temporal scalers](https://github.com/Nixtla/neuralforecast/blob/main/neuralforecast/common/_scalers.py). | 'identity' | | `random_seed` | [int](#int) | random\_seed for pytorch initializer and numpy generators. | 1 | | `drop_last_loader` | [bool](#bool) | if True `TimeSeriesDataLoader` drops last non-full batch. | False | | `alias` | [str](#str) | optional, Custom name of the model. | None | | `optimizer` | Subclass of 'torch.optim.Optimizer' | optional, user specified optimizer instead of the default choice (Adam). | None | | `optimizer_kwargs` | [dict](#dict) | optional, list of parameters used by the user specified `optimizer`. | None | | `lr_scheduler` | Subclass of 'torch.optim.lr\_scheduler.LRScheduler' | optional, user specified lr\_scheduler instead of the default choice (StepLR). | None | | `lr_scheduler_kwargs` | [dict](#dict) | optional, list of parameters used by the user specified `lr_scheduler`. | None | | `dataloader_kwargs` | [dict](#dict) | optional, list of parameters passed into the PyTorch Lightning dataloader by the `TimeSeriesDataLoader`. | None | | `**trainer_kwargs` | [int](#int) | keyword trainer arguments inherited from [PyTorch Lighning's trainer](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer). | {} |
References * [Chen, Si-An, Chun-Liang Li, Nate Yoder, Sercan O. Arik, and Tomas Pfister (2023). "TSMixer: An All-MLP Architecture for Time Series Forecasting."](http://arxiv.org/abs/2303.06053)
#### `TSMixer.fit` ```python theme={null} fit( dataset, val_size=0, test_size=0, random_seed=None, distributed_config=None ) ``` Fit. The `fit` method, optimizes the neural network's weights using the initialization parameters (`learning_rate`, `windows_batch_size`, ...) and the `loss` function as defined during the initialization. Within `fit` we use a PyTorch Lightning `Trainer` that inherits the initialization's `self.trainer_kwargs`, to customize its inputs, see [PL's trainer arguments](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer). The method is designed to be compatible with SKLearn-like classes and in particular to be compatible with the StatsForecast library. By default the `model` is not saving training checkpoints to protect disk memory, to get them change `enable_checkpointing=True` in `__init__`. **Parameters:** | Name | Type | Description | Default | | ------------- | ---------------------------------------------------- | -------------------------------------------------------------------------------------- | ----------------- | | `dataset` | [TimeSeriesDataset](#TimeSeriesDataset) | NeuralForecast's `TimeSeriesDataset`, see [documentation](./tsdataset.html). | *required* | | `val_size` | [int](#int) | Validation size for temporal cross-validation. | 0 | | `random_seed` | [int](#int) | Random seed for pytorch initializer and numpy generators, overwrites model.**init**'s. | None | | `test_size` | [int](#int) | Test size for temporal cross-validation. | 0 | **Returns:** | Type | Description | | ---- | ----------- | | None | | #### `TSMixer.predict` ```python theme={null} predict( dataset, test_size=None, step_size=1, random_seed=None, quantiles=None, h=None, explainer_config=None, **data_module_kwargs ) ``` Predict. Neural network prediction with PL's `Trainer` execution of `predict_step`. **Parameters:** | Name | Type | Description | Default | | ---------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------- | | `dataset` | [TimeSeriesDataset](#TimeSeriesDataset) | NeuralForecast's `TimeSeriesDataset`, see [documentation](./tsdataset.html). | *required* | | `test_size` | [int](#int) | Test size for temporal cross-validation. | None | | `step_size` | [int](#int) | Step size between each window. | 1 | | `random_seed` | [int](#int) | Random seed for pytorch initializer and numpy generators, overwrites model.**init**'s. | None | | `quantiles` | [list](#list) | Target quantiles to predict. | None | | `h` | [int](#int) | Prediction horizon, if None, uses the model's fitted horizon. Defaults to None. | None | | `explainer_config` | [dict](#dict) | configuration for explanations. | None | | `**data_module_kwargs` | [dict](#dict) | PL's TimeSeriesDataModule args, see [documentation](https://pytorch-lightning.readthedocs.io/en/1.6.1/extensions/datamodules.html#using-a-datamodule). | {} | **Returns:** | Type | Description | | ---- | ----------- | | None | | ### Usage Examples Train model and forecast future values with `predict` method. ```python theme={null} import pandas as pd import matplotlib.pyplot as plt from neuralforecast import NeuralForecast from neuralforecast.models import TSMixer from neuralforecast.utils import AirPassengersPanel, AirPassengersStatic from neuralforecast.losses.pytorch import MAE, MQLoss Y_train_df = AirPassengersPanel[AirPassengersPanel.ds=AirPassengersPanel['ds'].values[-12]].reset_index(drop=True) # 12 test model = TSMixer(h=12, input_size=24, n_series=2, n_block=4, ff_dim=4, dropout=0, revin=True, scaler_type='standard', max_steps=500, early_stop_patience_steps=-1, val_check_steps=5, learning_rate=1e-3, loss=MQLoss(), batch_size=32 ) fcst = NeuralForecast(models=[model], freq='ME') fcst.fit(df=Y_train_df, static_df=AirPassengersStatic, val_size=12) forecasts = fcst.predict(futr_df=Y_test_df) # Plot predictions fig, ax = plt.subplots(1, 1, figsize = (20, 7)) Y_hat_df = forecasts.reset_index(drop=False).drop(columns=['unique_id','ds']) plot_df = pd.concat([Y_test_df, Y_hat_df], axis=1) plot_df = pd.concat([Y_train_df, plot_df]) plot_df = plot_df[plot_df.unique_id=='Airline2'].drop('unique_id', axis=1) plt.plot(plot_df['ds'], plot_df['y'], c='black', label='True') plt.plot(plot_df['ds'], plot_df['TSMixer-median'], c='blue', label='median') plt.fill_between(x=plot_df['ds'][-12:], y1=plot_df['TSMixer-lo-90'][-12:].values, y2=plot_df['TSMixer-hi-90'][-12:].values, alpha=0.4, label='level 90') ax.set_title('AirPassengers Forecast', fontsize=22) ax.set_ylabel('Monthly Passengers', fontsize=20) ax.set_xlabel('Year', fontsize=20) ax.legend(prop={'size': 15}) ax.grid() ``` Using `cross_validation` to forecast multiple historic values. ```python theme={null} fcst = NeuralForecast(models=[model], freq='M') forecasts = fcst.cross_validation(df=AirPassengersPanel, static_df=AirPassengersStatic, n_windows=2, step_size=12) # Plot predictions fig, ax = plt.subplots(1, 1, figsize = (20, 7)) Y_hat_df = forecasts.loc['Airline1'] Y_df = AirPassengersPanel[AirPassengersPanel['unique_id']=='Airline1'] plt.plot(Y_df['ds'], Y_df['y'], c='black', label='True') plt.plot(Y_hat_df['ds'], Y_hat_df['TSMixer-median'], c='blue', label='Forecast') ax.set_title('AirPassengers Forecast', fontsize=22) ax.set_ylabel('Monthly Passengers', fontsize=20) ax.set_xlabel('Year', fontsize=20) ax.legend(prop={'size': 15}) ax.grid() ``` ## 2. Auxiliary Functions ### 2.1 Mixing layers A mixing layer consists of a sequential time- and feature Multi Layer Perceptron ([`MLP`](./models.mlp.html#mlp)). ### `MixingLayer` ```python theme={null} MixingLayer(n_series, input_size, dropout, ff_dim) ``` Bases: [Module](#torch.nn.Module) MixingLayer ### `FeatureMixing` ```python theme={null} FeatureMixing(n_series, input_size, dropout, ff_dim) ``` Bases: [Module](#torch.nn.Module) FeatureMixing ### `TemporalMixing` ```python theme={null} TemporalMixing(n_series, input_size, dropout) ``` Bases: [Module](#torch.nn.Module) TemporalMixing