Transformer-Based Probabilistic Time Series Forecasting Methodology

Time series forecasting is evolving away from traditional local approaches like ARIMA toward global models that learn latent representations from multiple time series. In particular, probabilistic forecasting, which goes beyond simple point forecasting to provide the uncertainty of predictions, is critically important in practical decision-making processes.

The Transformer architecture is highly suitable for time series forecasting for the following reasons:

• Encoder-Decoder structure: Similar to text generation, it can perform forecasting up to the desired prediction horizon through Ancestral Sampling, which predicts the next step by sampling from the predicted distribution.
• Causal-masked Decoder: By applying the Teacher Forcing technique during training, it effectively learns the sequential nature of time series.
• Handling missing values: By leveraging the Masking function of the attention mechanism, time series data containing missing values can be directly used for training without a separate data imputation process.