LG AI Research 387

Tabular Data is data structured in table form composed of rows and columns, and compared to unstructured data such as images, text, and speech, it is difficult to achieve good performance when applying deep learning to it. This is because deep networks are specialized for learning complex and smooth functions, which limits their ability to learn the Irregular Function characteristic of tabular data's discontinuous features.

To overcome this, LG AI Research's DI Lab proposed a methodology called Binning as a Pretext Task. This approach utilizes a Binning Algorithm that converts Continuous variables into Discrete variables, and is designed—through an Autoencoder structure—to predict the Binning Class of an input value instead of the input value itself.

The main advantages of this approach are as follows:

• Providing Inductive Bias: By changing the target into a discrete variable, it helps with learning the Irregular Function.
• Data Integration: In environments where numerical and discrete variables are mixed, all variables are set as Bins, allowing them to be handled effectively.
• Noise Suppression: Similar values can be grouped together, allowing minor errors to be ignored.
• Flexibility: It can be freely combined with existing Encoder networks, Augmentation, or Loss Functions.

Experimental results showed that this methodology delivered excellent performance on Binary/Multiclass Classification and Regression tasks across 25 datasets, achieving performance comparable to or better than state-of-the-art Tree-based algorithms.

This article also introduces other research presented at NeurIPS 2023. In particular, it covers a large-scale study analyzing the performance differences between GBDTs (CatBoost, LightGBM, XGBoost) and Neural Networks (TabPFN, FT-Transformer, etc.), suggesting that GBDTs still show strong performance on irregular data or large-scale datasets.