Research on Representation Space Augmentation for Effective Self-Supervised Learning of Tabular Data

LG AI Research's DI Lab unveiled a new methodology called RaTab (Representation Space Augmentation for Tabular Data) at AAAI-25, enabling effective learning of Tabular Data, which is essential for manufacturing processes and quality control.

Existing approaches to tabular data learning relied on tree-based models such as XGBoost, or applied self-supervised learning (SSL) techniques optimized for images/text. However, existing SSL data augmentation techniques (Masking, Shuffling, etc.) had limitations in that they distorted the relationships between key features in tabular data or caused information loss.

RaTab performs augmentation in the Representation Space rather than the Input Space to address these issues. Its key features are as follows:

• Representation Space Augmentation: Transforms the weight matrix of the encoder's last MLP layer to generate more meaningful representations.
• Utilizing Truncated SVD: Decomposes the weight matrix using SVD (Singular Value Decomposition) to secure diversity in representations while preserving the data's Core Information.
• Preserving Data Structure: Unlike existing methods, it enables effective representation learning without distorting complex interactions between features.

This research is significant in that it lays a powerful deep learning-based foundation that can contribute to process optimization and productivity improvement by processing vast sensor data and log data from manufacturing sites more precisely.