​

module neuralforecast.models.timellm

​

Global Variables

• IS_TRANSFORMERS_INSTALLED

​

class ReplicationPad1d

​

method __init__

__init__(padding)

​

method forward

forward(input)

​

class TokenEmbedding

​

method __init__

__init__(c_in, d_model)

​

method forward

forward(x)

​

class PatchEmbedding

​

method __init__

__init__(d_model, patch_len, stride, dropout)

​

method forward

forward(x)

​

class FlattenHead

​

method __init__

__init__(n_vars, nf, target_window, head_dropout=0)

​

method forward

forward(x)

​

class ReprogrammingLayer

​

method __init__

__init__(d_model, n_heads, d_keys=None, d_llm=None, attention_dropout=0.1)

​

method forward

forward(target_embedding, source_embedding, value_embedding)

​

method reprogramming

reprogramming(target_embedding, source_embedding, value_embedding)

​

class TimeLLM

• h (int): Forecast horizon.
• input_size (int): autorregresive inputs size, y=[1,2,3,4] input_size=2 -> y_[t-2:t]=[1,2].
• patch_len (int): length of patch. Default: 16
• stride (int): stride of patch. Default: 8
• d_ff (int): dimension of fcn. Default: 128
• top_k (int): top tokens to consider. Default: 5
• d_llm (int): hidden dimension of LLM. Default: 768 # LLama7b:4096; GPT2-small:768; BERT-base:768
• d_model (int): dimension of model. Default: 32
• n_heads (int): number of heads in attention layer. Default: 8
• enc_in (int): encoder input size. Default: 7
• dec_in (int): decoder input size. Default: 7
• llm (str): Path to pretrained LLM model to use. If not specified, it will use GPT-2 from https://huggingface.co/openai-community/gpt2”
  
• llm_config (dict): Deprecated, configuration of LLM. If not specified, it will use the configuration of GPT-2 from https://huggingface.co/openai-community/gpt2”
  
• llm_tokenizer (str): Deprecated, tokenizer of LLM. If not specified, it will use the GPT-2 tokenizer from https://huggingface.co/openai-community/gpt2”
  
• llm_num_hidden_layers (int): hidden layers in LLM. Default: 32
• llm_output_attention (bool): whether to output attention in encoder. Default: True
• llm_output_hidden_states (bool): whether to output hidden states. Default: True
• prompt_prefix (str): prompt to inform the LLM about the dataset. Default: None
• dropout (float): dropout rate. Default: 0.1
• stat_exog_list (list): static exogenous columns.
• hist_exog_list (list): historic exogenous columns.
• futr_exog_list (list): future exogenous columns.
• loss (PyTorch module): instantiated train loss class from losses collection.
• valid_loss (PyTorch module): instantiated valid loss class from losses collection.
• learning_rate (float): Learning rate between (0, 1). Default: 1e-3
• max_steps (int): maximum number of training steps. Default: 1000
• val_check_steps (int): Number of training steps between every validation loss check. Default: 100
• batch_size (int): number of different series in each batch. Default: 32
• valid_batch_size (int): number of different series in each validation and test batch, if None uses batch_size. Default: None
• windows_batch_size (int): number of windows to sample in each training batch, default uses all. Default: 1024
• inference_windows_batch_size (int): number of windows to sample in each inference batch. Default: 1024
• start_padding_enabled (bool): if True, the model will pad the time series with zeros at the beginning, by input size. Default: False
• training_data_availability_threshold (Union[float, List[float]]): minimum fraction of valid data points required for training windows. Single float applies to both insample and outsample; list of two floats specifies [insample_fraction, outsample_fraction]. Default 0.0 allows windows with only 1 valid data point (current behavior).
  
• step_size (int): step size between each window of temporal data. Default: 1
• num_lr_decays (int): Number of learning rate decays, evenly distributed across max_steps. Default: -1
• early_stop_patience_steps (int): Number of validation iterations before early stopping. Default: -1
• scaler_type (str): type of scaler for temporal inputs normalization see temporal scalers. Default: ‘identity’
• random_seed (int): random_seed for pytorch initializer and numpy generators. Default: 1
• drop_last_loader (bool): if True TimeSeriesDataLoader drops last non-full batch. Default: False
• alias (str): optional, Custom name of the model.
• optimizer (Subclass of ‘torch.optim.Optimizer’): optional, user specified optimizer instead of the default choice (Adam).
• optimizer_kwargs (dict): optional, list of parameters used by the user specified optimizer.
• lr_scheduler (Subclass of ‘torch.optim.lr_scheduler.LRScheduler’): optional, user specified lr_scheduler instead of the default choice (StepLR).
• lr_scheduler_kwargs (dict): optional, list of parameters used by the user specified lr_scheduler.
• dataloader_kwargs (dict): optional, list of parameters passed into the PyTorch Lightning dataloader by the TimeSeriesDataLoader.
• **trainer_kwargs (int): keyword trainer arguments inherited from PyTorch Lighning’s trainer.

• Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, Qingsong Wen. “Time-LLM: Time Series Forecasting by Reprogramming Large Language Models”

​

method __init__

__init__(
    h,
    input_size,
    patch_len: int = 16,
    stride: int = 8,
    d_ff: int = 128,
    top_k: int = 5,
    d_llm: int = 768,
    d_model: int = 32,
    n_heads: int = 8,
    enc_in: int = 7,
    dec_in: int = 7,
    llm=None,
    llm_config=None,
    llm_tokenizer=None,
    llm_num_hidden_layers=32,
    llm_output_attention: bool = True,
    llm_output_hidden_states: bool = True,
    prompt_prefix: Optional[str] = None,
    dropout: float = 0.1,
    stat_exog_list=None,
    hist_exog_list=None,
    futr_exog_list=None,
    loss=MAE(),
    valid_loss=None,
    learning_rate: float = 0.0001,
    max_steps: int = 5,
    val_check_steps: int = 100,
    batch_size: int = 32,
    valid_batch_size: Optional[int] = None,
    windows_batch_size: int = 1024,
    inference_windows_batch_size: int = 1024,
    start_padding_enabled: bool = False,
    training_data_availability_threshold=0.0,
    step_size: int = 1,
    num_lr_decays: int = 0,
    early_stop_patience_steps: int = -1,
    scaler_type: str = 'identity',
    random_seed: int = 1,
    drop_last_loader: bool = False,
    alias: Optional[str] = None,
    optimizer=None,
    optimizer_kwargs=None,
    lr_scheduler=None,
    lr_scheduler_kwargs=None,
    dataloader_kwargs=None,
    **trainer_kwargs
)

​

property automatic_optimization

​

property current_epoch

​

property device

​

property device_mesh

​

property dtype

​

property example_input_array

• Single tensor: It is assumed the model takes a single argument, i.e., model.forward(model.example_input_array)
• Tuple: The input array should be interpreted as a sequence of positional arguments, i.e., model.forward(*model.example_input_array)
• Dict: The input array represents named keyword arguments, i.e., model.forward(**model.example_input_array)

​

property fabric

​

property global_rank

​

property global_step

​

property hparams

​

property hparams_initial

• AttributeDict: immutable initial hyperparameters

​

property local_rank

​

property logger

​

property loggers

​

property on_gpu

​

property strict_loading

​

property trainer

​

method calcute_lags

calcute_lags(x_enc)

​

method forecast

forecast(x_enc)

​

method forward

forward(windows_batch)