Different forecasts for same time series w/ NBEATS

[MeirYosub]

Hi there,
In addition to the statistical forecasting using the ARIMA and ETS models I try to implement forecasting using neural networks.
I encountered an unexplained matter that when I make a forecast with the NBEATS model for a single time series with a certain unique_id I get completely different forecasting results than those obtained when the forecast is made for the same series and others with different unique_id at the same time.
What is the reason for this?

I must point out that Nixtla has done a very impressive job in building and implementing the models for predicting time series and there is no doubt that there is a significant contribution in capabilities to the entire community.
Huge thumbs up for that.

[elephaint]

Hi, thanks for using neuralforecast and thanks for the kind words :)

Unfortunately without a standalone piece of code that I can run myself that exposes the issue it's difficult to help you; it may be an error in Neuralforecast or NBEATS, or an error in your code / training pipeline, it's near-impossible to tell without more information. So, if you can provide a minimal example that reproduces this behaviour with NBEATS I can help you further.

[MeirYosub]

code demo and the results

data

model fit and forecast

subset the data to only 2 unique_id series

getting different forecast

[MeirYosub]

Hi there,
Anything new on this topic?

[elephaint]

Hi - it's hard to judge from pictures (a standalone piece of code that I can run myself would help).

That said, first you are training a model on many series, thereafter only on 2. So obviously you will get different results (?)

[MeirYosub]

Hi there
I am adding code (jupyter notebook as txt file) + data

NBEATS_demo (copy).txt
data.csv

[MeirYosub]

Something here is not clear to me.
Why forecasting several series simultaneously with the NBEATS model should change the forecast result of a specific series?
After all, each series is actually separate from the others and is identified by a unique key.
This behavior does not exist when forecasting with statistical models such as ETS and autoARIMA.

[MeirYosub]

Any new comment on the issue ?

[MeirYosub]

From what I am able to understand from the results: in the case of several time series, one and only neural model (NBEATS) is adjusted for all the series, in contrast to the statistical models in which a separate model is adjusted for each series.

[elephaint]

@MeirYosub Your understanding is (mostly) correct.

If you are training a single neural network on multiple series, you get a single model that can produce forecasts for multiple series. In statistical methods we typically train a single model per series. A key motivation of using a neural network is to leverage its superior representation capacity to learn a single model across many series such that it can be used for forecasting multiple series. Training a neural network on a single series only (and thus having separate models per series) kind of destroys the entire reason of using a neural network.

I'm still not sure there is an issue here (it seems mostly a question of understanding the basic concepts of forecasting with neural networks). Do you have the feeling you understand these concepts? If not, happy to provide further clarifications. If so, I think we can close this issue. Let me know :)

[MeirYosub]

Many thanks for the explanation. The issue seems to have been understood.