NVIDIA Releases Kumo Tabular: Open Foundation Model for Zero-Shot Tabular Prediction

NVIDIA has released Kumo Tabular, an open foundation model for tabular data that performs classification and regression in a single forward pass without training, tuning, or feature engineering. As part of the NVIDIA Kumo Structured model collection, it applies in-context learning principles from LLMs to tables, allowing pre-trained models to read labeled tables as context and predict new rows without weight updates.

Architecture and Training

The model utilizes a Transformer structure based on column, row, and in-context attention mechanisms similar to TabICL and TabPFN. Key architectural features include:

• Cell Embedding: Uses Fourier features for numerical/categorical values and special handling for missing values without imputation.
• Row Embedding: Combines column attention (to understand value distributions) and row attention (to learn token interactions within a row).
• In-context Learning: Context rows attend to each other, while query rows attend only to context rows, enabling efficient reuse of context keys/values.
• Length-aware Attention Temperature: Scales attention temperature logarithmically with the number of keys to maintain sharp attention as table size increases.

Training relies exclusively on artificial data generated via Structural Causal Models (SCM). The process involves sampling table structures, connecting hidden variables via random causal graphs, and injecting realistic flaws like missing patterns and outliers. Three model sizes are available (28M to 215M parameters), trained on approximately 35 to 137 million artificial tables respectively.

Performance and Benchmarks

Evaluated on a single RTX 6000 Pro, Kumo Tabular sets a new accuracy-efficiency frontier:

• TabArena: Ranked 1st overall with an ELO of 1950, achieving 17x faster inference than LimiX-2.
• BeyondArena: Ranked 1st overall with an ELO of 1418.
• TALENT: Ranked 1st overall in classification accuracy, log-loss, and regression RMSE.
• ScoringBench: Kumo Tabular-Large ranked 1st in predictive distribution benchmarks.

Limitations and Usage

The model supports numerical and categorical columns directly; text, images, and timestamps require feature transformation via built-in preprocessing recipes. Single forward passes support up to 10 classes for classification, with error-correcting output codes used for higher counts. Users must validate accuracy and calibration on held-out data before deployment, especially if query distributions differ from context.

The model is available under the OpenMDW-1.1 license (commercial use permitted) via the structured-data-models GitHub repository and Hugging Face weights. It is accessed through NVIDIA's GPU-native sdm library, which handles preprocessing, ensembling, and weight downloading.