Controlling the level of fine-tuning
1. Import packages
First, we import the required packages and initialize the Nixtla client
👍 Use an Azure AI endpoint
To use an Azure AI endpoint, remember to set also the
base_urlargument:
nixtla_client = NixtlaClient(base_url="you azure ai endpoint", api_key="your api_key")
2. Load data
| timestamp | value | |
|---|---|---|
| 0 | 1949-01-01 | 112 |
| 1 | 1949-02-01 | 118 |
| 2 | 1949-03-01 | 132 |
| 3 | 1949-04-01 | 129 |
| 4 | 1949-05-01 | 121 |
Now, we split the data into a training and test set so that we can
measure the performance of the model as we vary finetune_depth.
Next, we fine-tune TimeGPT and vary finetune_depth to measure the
impact on performance.
3. Fine-tuning with finetune_depth
📘 Available models in Azure AI
If you are using an Azure AI endpoint, please be sure to set
model="azureai":
nixtla_client.forecast(..., model="azureai")For the public API, we support two models:
timegpt-1andtimegpt-1-long-horizon.By default,
timegpt-1is used. Please see this tutorial on how and when to usetimegpt-1-long-horizon.
As mentioned above, finetune_depth controls how many parameters from
TimeGPT are fine-tuned on your particular dataset. If the value is set
to 1, only a few parameters are fine-tuned. Setting it to 5 means that
all parameters of the model will be fine-tuned.
Using a large value for finetune_depth can lead to better performances
for large datasets with complex patterns. However, it can also lead to
overfitting, in which case the accuracy of the forecasts may degrade, as
we will see from the small experiment below.
| unique_id | metric | TimeGPT_depth1 | TimeGPT_depth2 | TimeGPT_depth3 | TimeGPT_depth4 | TimeGPT_depth5 | |
|---|---|---|---|---|---|---|---|
| 0 | 0 | mae | 22.675540 | 17.908963 | 21.318518 | 24.745096 | 28.734302 |
| 1 | 0 | mse | 677.254283 | 461.320852 | 676.202126 | 991.835359 | 1119.722602 |
From the result above, we can see that a finetune_depth of 2 achieves
the best results since it has the lowest MAE and MSE.
Also notice that with a finetune_depth of 4 and 5, the performance
degrades, which is a clear sign of overfitting.
Thus, keep in mind that fine-tuning can be a bit of trial and error. You
might need to adjust the number of finetune_steps and the level of
finetune_depth based on your specific needs and the complexity of your
data. Usually, a higher finetune_depth works better for large
datasets. In this specific tutorial, since we were forecasting a single
series with a very short dataset, increasing the depth led to
overfitting.
It’s recommended to monitor the model’s performance during fine-tuning
and adjust as needed. Be aware that more finetune_steps and a larger
value of finetune_depth may lead to longer training times and could
potentially lead to overfitting if not managed properly.