Using Binning as a Pretext Task: Improving Self-Supervised Learning in the Tabular Domain

LG AI Research's Data Intelligence (DI) Lab presented the paper 'Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domains' at ICML 2024. This research overcomes the limitations of tabular data learning by using a Binning algorithm that converts numerical variables into categorical variables, performing a Pretext Task that predicts the corresponding Bin index during unsupervised learning.

Unlike the earlier workshop presentation, this paper demonstrates performance using T2G-Former, a state-of-the-art tabular learning model, and proves the effectiveness of the methodology through various experimental results and visualizations. Of the 2,634 papers presented at ICML 2024, only 14 were tabular domain-specific studies, and most of these studies target specific problem settings (small sample sizes, anomaly detection, etc.).

Another study introduced alongside this one, CHARMS (Channel Tabular Alignment with Optimal Transport), addresses multimodal learning between image and tabular data. This methodology maximizes Mutual Information between image channels and tabular attributes to effectively align data with different modalities and dimensions.

Key process of CHARMS:

• Image data: Representations are extracted using ResNet, and channels are grouped via K-Means clustering to generate $C'$ channels with independent meanings.
• Tabular data: Representations for each attribute are extracted using FT-Transformer.
• Alignment: The two modalities are aligned by measuring similarity between image channels and tabular attributes.