Mistral AI releases Timestral, a family of open-source time series foundation models

Mistral AI has released Timestral, a family of time series foundation models designed for zero-shot forecasting. The open-source release includes checkpoints in 5M, 28M, 123M, and 209M parameter sizes, with source code available on GitHub to facilitate reproduction of benchmark results.

Model Architecture and Capabilities

The models utilize a patch-based transformer architecture featuring time attention and variate attention. Key technical specifications include:

• Architecture: 24 layers with an embedding dimension of 768.
• Patching: Processes 32 time steps per patch, embedding values alongside missing-value indicators.
• Context Window: Accepts up to 32,768 historical time steps (1,024 patches).
• Forecast Horizon: Generates up to 1,024 steps in a single forward pass; longer horizons are handled iteratively by forecasting block-by-block.

Forecasting Features

Timestral supports both univariate and multivariate forecasting, accommodating past-only and known-future covariates (numeric or categorical). It provides probabilistic predictions across 21 quantiles (ranging from 0.01 to 0.99). Missing values are handled via NaN markers, and inference is supported on CPU or GPU using PyTorch.