LG AI Research Presents VarDrop Study at AAAI-25 to Improve Time-Series Forecasting Efficiency

Recent Transformer models in the time-series forecasting field use a Variate Tokenization strategy that treats each variable as a separate token. However, as the number of variables increases, computational cost grows exponentially, making it difficult to apply existing efficient training strategies.

LG AI Research's Data Intelligence (DI) Lab proposed the VarDrop algorithm to address this problem. This research focused on 'Variate Redundancy,' where variables within time-series data show similar patterns to one another.

The core mechanism of VarDrop is as follows:

• Adaptive Grouping: In response to the Covariate Shift phenomenon where data distribution changes, redundant variables are grouped on a per-batch basis.
• Efficient Sampling: Instead of training on all variables, only representative variables from each group are selectively utilized to maximize training efficiency.
• k-DFH Algorithm: Through k-Dominant Frequency Hashing utilizing Fourier transform, variables are grouped much faster than with existing clustering methods.

This approach dramatically reduces the number of variables required for training while minimizing information loss, enabling efficient time-series forecasting model training even in large-scale service environments.