Time Series Foundation Models for Forecasting Task

From late 2024 to early 2025, a wave of new time-series foundation models was released by major vendors like Amazon, Google, Salesforce, and others. These latest zero-shot foundational models for time series forecasting — including TimesFM (Google), Chronos (Amazon), Moirai (Salesforce), TimeGPT, Lag-LLama (ServiceNow) and Timer-XL (THUML) — which mark a significant leap forward in the field of time series forecasting models.

figure: time-series forecast foundation model architectures

The architecture of these foundation models includes the following flavors:

• Decoder models: TimesFM (Google), Timer-XL(THUML)
• Encoder models: MOIRAI (Salesforce)
• Encoder-Decoder: Chronos (Amazon)

For more information about Transformer architecture, please visit: https://huggingface.co/learn/llm-course/en/chapter1/6

Comparing foundation models is a daunting task because they are not exactly the same. Current benchmark results show that decoder models are more effective, with decoder-only and encoder-decoder models outperforming encoders in forecasting tasks. However, for tasks requiring general time-series understanding (eg. anomaly detection), encoder models may be more suitable.

The promise of these innovative, specialized foundation models for time-series forecasting tasks is to…