TabImpute: Accurate and Fast Zero-Shot Missing-Data Imputation with a Pre-Trained Transformer

• URL: http://arxiv.org/abs/2510.02625v2
• Date: Fri, 10 Oct 2025 15:55:12 GMT
• Title: TabImpute: Accurate and Fast Zero-Shot Missing-Data Imputation with a Pre-Trained Transformer
• Abstract summary: TabImpute is a pre-trained transformer that delivers accurate and fast zero-shot imputations.<n>MissBench is a benchmark for evaluation of imputation methods with $42$ OpenML datasets and $13$ missingness patterns.
• Score: 10.73832903466491
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Missing data is a pervasive problem in tabular settings. Existing solutions range from simple averaging to complex generative adversarial networks. However, due to huge variance in performance across real-world domains and time-consuming hyperparameter tuning, no default imputation method exists. Building on TabPFN, a recent tabular foundation model for supervised learning, we propose TabImpute, a pre-trained transformer that delivers accurate and fast zero-shot imputations requiring no fitting or hyperparameter tuning at inference-time. To train and evaluate TabImpute, we introduce (i) an entry-wise featurization for tabular settings, which enables a $100\times$ speedup over the previous TabPFN imputation method, (ii) a synthetic training data generation pipeline incorporating realistic missingness patterns, which boosts test-time performance, and (iii) MissBench, a comprehensive benchmark for evaluation of imputation methods with $42$ OpenML datasets and $13$ missingness patterns. MissBench spans domains such as medicine, finance, and engineering, showcasing TabImpute's robust performance compared to $11$ established imputation methods.

Related papers

• TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction [8.596879252293617]
  Tabular foundation models, driven by in-context learning, have rapidly grown in quality and popularity.<n>We present TabDPT-Turbo, a model that provides comparable default performance to TabDPT v1.1 on Tab-Lite, CC18, and CTR23, at orders of magnitude faster.
  arXiv  Detail & Related papers  (2026-08-02T17:33:30Z)
• Is TabPFN the Silver Bullet for Insurance Pricing? [4.5497948012757865]
  Tabular foundation models (TFMs) present a fundamentally different inference paradigm.<n>By pre-training on large collections of synthetic datasets, TFMs enable inference on new data through in-context learning.<n>This paper presents a first empirical evaluation of TabPFN for motor insurance pricing.
  arXiv  Detail & Related papers  (2026-05-21T13:59:06Z)
• Test time training enhances in-context learning of nonlinear functions [51.56484100374058]
  Test-time training (TTT) enhances model performance by explicitly updating designated parameters prior to each prediction.<n>We investigate the combination of TTT with in-context learning (ICL), where the model is given a few examples from the target distribution at inference time.
  arXiv  Detail & Related papers  (2025-09-30T03:56:44Z)
• Pre-training under infinite compute [87.02472603429936]
  We show that data-constrained approaches of increasing epoch count and parameter count eventually overfit.<n>We identify that ensembling independently trained models achieves a significantly lower loss asymptote than the regularized recipe.<n>Our results show that simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future.
  arXiv  Detail & Related papers  (2025-09-18T09:36:23Z)
• Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation [59.1067331268383]
  Test-Time Adaptation (TTA) has emerged as an effective solution for adapting Vision Transformers (ViT) to distribution shifts without additional training data.<n>To reduce inference cost, plug-and-play token aggregation methods merge redundant tokens in ViTs to reduce total processed tokens.<n>We formalize this problem as Efficient Test-Time Adaptation (ETTA), seeking to preserve the adaptation capability of TTA while reducing inference latency.
  arXiv  Detail & Related papers  (2025-08-05T12:40:55Z)
• Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer [0.0]
  This article proposes an imputation-free incremental attention learning (IFIAL) method for tabular data.<n>A pair of attention masks is derived and retrofitted to a transformer to directly streamline data without imputing or initializing missing values.<n>Experiments substantiate the robustness of IFIAL against varying missing value types and rates compared to methods involving missing value imputation.
  arXiv  Detail & Related papers  (2025-04-20T13:31:49Z)
• Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer [2.1677183904102257]
  We present an Adversarially Pre-trained Transformer (APT) that is able to perform zero-shot meta-learning on tabular prediction tasks without pre-training on any real-world dataset.<n>APT is pre-trained with adversarial synthetic data agents, who deliberately challenge the model with different synthetic datasets.<n>We show that our framework matches state-of-the-art performance on small classification tasks without filtering on dataset characteristics.
  arXiv  Detail & Related papers  (2025-02-06T23:58:11Z)
• DispFormer: A Pretrained Transformer Incorporating Physical Constraints for Dispersion Curve Inversion [56.64622091009756]
  This study introduces DispFormer, a transformer-based neural network for $v_s$ profile inversion from Rayleigh-wave phase and group dispersion curves.<n>DispFormer processes dispersion data independently at each period, allowing it to handle varying lengths without requiring network modifications or strict alignment between training and testing datasets.
  arXiv  Detail & Related papers  (2025-01-08T09:08:24Z)
• APAR: Modeling Irregular Target Functions in Tabular Regression via Arithmetic-Aware Pre-Training and Adaptive-Regularized Fine-Tuning [12.35924469567586]
  We propose a novel Arithmetic-Aware Pre-training and Adaptive-Regularized Fine-tuning framework (APAR)<n>In the pre-training phase, APAR introduces an arithmetic-aware pretext objective to capture intricate sample-wise relationships from the perspective of continuous labels.<n>In the fine-tuning phase, a consistency-based adaptive regularization technique is proposed to self-learn appropriate data augmentation.
  arXiv  Detail & Related papers  (2024-12-14T19:33:21Z)
• Interpretable Machine Learning for TabPFN [5.012821694203072]
  The TabPFN model is able to achieve state-of-the-art performance on a variety of classification tasks. By taking advantage of the unique properties of the model, our adaptations allow for more efficient computations.
  arXiv  Detail & Related papers  (2024-03-16T13:35:15Z)
• TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second [48.87527918630822]
  We present TabPFN, a trained Transformer that can do supervised classification for small datasets in less than a second. TabPFN performs in-context learning (ICL), it learns to make predictions using sequences of labeled examples. We show that our method clearly outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 230$times$ speedup.
  arXiv  Detail & Related papers  (2022-07-05T07:17:43Z)
• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning [81.3514358542452]
  Few-shot in-context learning (ICL) incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made. parameter-efficient fine-tuning offers an alternative paradigm where a small set of parameters are trained to enable a model to perform the new task. In this paper, we rigorously compare few-shot ICL and parameter-efficient fine-tuning and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs.
  arXiv  Detail & Related papers  (2022-05-11T17:10:41Z)
• Time-Series Imputation with Wasserstein Interpolation for Optimal Look-Ahead-Bias and Variance Tradeoff [66.59869239999459]
  In finance, imputation of missing returns may be applied prior to training a portfolio optimization model. There is an inherent trade-off between the look-ahead-bias of using the full data set for imputation and the larger variance in the imputation from using only the training data. We propose a Bayesian posterior consensus distribution which optimally controls the variance and look-ahead-bias trade-off in the imputation.
  arXiv  Detail & Related papers  (2021-02-25T09:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.