LG AI Research Releases 'EXAONE Tabular', a Lightweight Foundation Model for Tabular Data

LG AI Research has released 'EXAONE Tabular', a foundation model for classification and regression on structured (tabular) data. This model was developed to address the limitations of existing tree-based boosting methods (such as XGBoost and LightGBM), specifically the need for dataset-specific retraining and high resource consumption.

Core Architecture and Training Method

EXAONE Tabular was pre-trained solely on synthetic data based on Structural Causal Models (SCM), rather than real tabular data. This approach eliminates domain-specific dependencies and ensures generalization capabilities for unseen datasets.

• Cross-Axis Summary Transformer (CAST): Alternates attention between the feature-axis and sample-axis to refine feature interactions and support-set context.
• In-context Learning: Performs immediate predictions without parameter updates when provided with a labeled support set and an unlabeled query set.
• Robustness: Incorporates missing values and error patterns during pre-training, enabling stable predictions even when inputting data with missing values without separate imputation.

Performance and Efficiency

The model is highly lightweight, with 20.8M parameters for classification and 21.11M for regression. It achieved the overall top ranking on the TabArena benchmark without tuning, delivering performance comparable to Google's TabFM (1.64B parameters) while reducing inference costs to approximately 1/11.

• Classification Performance: Outperforms TabPFN-3 by approximately 125 Elo points and exceeds the performance of the AutoGluon AutoML pipeline.
• Regression Performance: Surpasses TabFM and demonstrates the ability to capture prediction uncertainty (CRPS metric) on ScoringBench.
• Inference Speed: Records low latency of 0.605 seconds per 1,000 samples, making it suitable for real-time deployment.

Deployment and Usage

The model is already deployed in manufacturing processes, such as battery cell defect detection, for product quality prediction and early warning of potential defects. LG AI Research has released inference software for research and educational purposes on GitHub and Hugging Face, while commercial use requires a separate license. The company plans to expand applications to the biotech, healthcare, and finance sectors.