[AAAI-25] Research on Improving Time Series Forecasting Model Training Efficiency by Mitigating Variate Redundancy - LG AI Research BLOG

The Variate Tokenization strategy, widely used recently in time series forecasting, has a limitation in that computational cost increases quadratically as the number of variates grows. In particular, high Variate Redundancy within multivariate time series data forces models to process unnecessary information, hindering training efficiency.

VarDrop, presented by LG AI Research's DI Lab at AAAI-25, is a new approach that adaptively groups these redundant variates on a per-batch basis and samples only representative variates for use in training. This algorithm operates in two main stages.

• k-Dominant Frequency Hashing (k-DFH): Applies Fourier transform and uses the top k frequencies as hash values, grouping variates at a much lower cost than existing clustering methods.
• Stratified Sampling: Extracts representative variates from groups of variates that share the same hash value for use in training.

VarDrop allows the trade-off between performance and cost to be controlled via the hyperparameters k (group resolution) and gs (number of samples within a group). Experimental results showed that, beyond simply reducing tokens, it achieves superior forecasting performance compared to random sampling while significantly reducing computation time and memory usage.