LG AI Research: Trends in Time Series Foundation Models

Time Series data plays a key role in fields where future prediction is important, such as weather, sensors, and finance. Recently in the AI field, active efforts are underway to build Time Series Foundation models that can be universally applied across various domains.

Time Series Foundation models are broadly classified into methodologies that utilize LLMs (Large Language Models) and methodologies that build their own large-scale models. Methodologies utilizing LLMs are further divided into three main types.

• Without Adaptation: This assumes that the LLM already possesses time series knowledge, and focuses on designing appropriate Prompts. A representative example is PromptCast, though issues with number tokenization (e.g., token mismatch between 480 and 481) have been pointed out. LLM-Time, which improves on this, secured performance without additional training by processing numbers as individual tokens.
• Adapt LLM: This presupposes that the LLM is not familiar with the time series domain, and adds an Adaptation Layer to Fine-tune the model. OFA utilizes Patching and a Linear Layer, while TEMPO improved performance by applying time series decomposition and LoRA.
• Adapt to LLM: This assumes that the LLM has 'time series knowledge converted into text'. The key is how to effectively convert time series data into text, and Time-LLM performs mutual conversion between time series and text using mechanisms such as Cross Attention.