TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

• URL: http://arxiv.org/abs/2607.11007v1
• Date: Mon, 13 Jul 2026 02:11:27 GMT
• Title: TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings
• Abstract summary: We evaluate TabPFN as a plug-and-play, zero-gradient classification head for frozen image, text, and audio encoders.<n>Across 22,820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log-likelihood (NLL) and expected calibration error (ECE)<n>Its accuracy advantage is conditional, concentrating at moderate-to-high shot counts and low-to-moderate feature dimensions.
• Score: 2.4483752182160954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot multimodal classification commonly attaches a lightweight head, such as $k$-nearest neighbors, logistic regression, or a linear SVM, to a frozen pretrained encoder. Although computationally efficient, these heads can produce poorly calibrated confidence scores, limiting their reliability in calibration-sensitive applications. We evaluate TabPFN as a plug-and-play, zero-gradient classification head for frozen image, text, and audio encoders. Across 22{,}820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log-likelihood (NLL) and expected calibration error (ECE). At a representative setting, it reduces NLL by 48--62\% and ECE by 2.1--5.3$\times$ relative to the average of the eight baselines while matching or exceeding their average accuracy. Its accuracy advantage is conditional, concentrating at moderate-to-high shot counts and low-to-moderate feature dimensions ($k \ge 50$, $d \le 32$), and diminishing when labeled data are scarce, feature dimensions are high, or competing methods approach ceiling accuracy. In targeted backbone-adaptation experiments, replacing the trained linear head with TabPFN substantially improves calibration while preserving competitive accuracy. These results provide empirical guidance for using TabPFN as a training-free head in calibration-sensitive multimodal classification. To support transparency and reproducibility, we publicly release the source code, experiment configurations, and evaluation scripts in our GitHub repository: https://github.com/Jingxiang-Zhang/tabpfn-multimodal-embeddings.

Related papers

• Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation [18.8104643146881]
  We study the choice of constructor-efficient fine-tuning under a severe trainable-state budget.<n>We introduce FCCA, which estimates the signed input--error cross-covariance, whitens it with diagonal Fisher moments, truncates it in the resulting local metric, maps the selected directions back, and applies thin QR to obtain stable core coordinates.<n>On Qwen2.5-3B, FCCA reaches an 83.0 macro-average, 2.3 points above the next-best matched-budget constructor, and exceeds its unwhitened RawGrad control on all 11 tasks.
  arXiv  Detail & Related papers  (2026-09-01T05:52:38Z)
• Invertible Logits Transformation for Accuracy-Preserving Post-Hoc Uncertainty Calibration [10.333478793903724]
  An ideal calibrator should correct nonlinear miscalibration, scale gracefully to large label spaces, and preserve the original predictions.<n>InvLT consistently outperforms a broad set of post-hoc baselines on standard calibration metrics.
  arXiv  Detail & Related papers  (2026-08-11T02:08:05Z)
• Accurate and Resource-Efficient Federated Continual Learning [3.983545455427237]
  Federated continual learning (FCL) must learn from distributed task streams under limited resources.<n>We propose FedRAN, a resource-aware analytic FCL framework that replaces gradient-based updates with compact random feature statistics.<n>FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6-121.8$times$ less per-client communication than optimization-based FCL, and is 190.3$times$ faster on average than gradient-based baselines.
  arXiv  Detail & Related papers  (2026-06-09T22:14:16Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• Lightweight Vulnerability Detection from Code Metrics and Token Features [0.0]
  This preprint studies a lightweight function-level vulnerability triage pipeline that combines sparse token n-grams from raw function text with a small set of inexpensive code metrics.<n>The results suggest that simple token and metric features provide a useful transparent baseline, but also expose sensitivity to superficial lexical cues and limited crossproject transfer.
  arXiv  Detail & Related papers  (2026-05-05T19:51:46Z)
• Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning [18.69627681731888]
  Online Reasoning calibration (ORCA) is a framework for calibrating the sampling process that draws upon conformal prediction and test-time training.<n>At risk level $=0.1$, ORCA improves Qwen2.5-32B efficiency on in-distribution tasks with savings up to 47.5% with supervised labels and 40.7% with self-consistency labels.
  arXiv  Detail & Related papers  (2026-04-01T17:21:50Z)
• Adaptive $k$-nearest neighbor classifier based on the local estimation of the shape operator [49.87315310656657]
  We introduce a new adaptive $k$-nearest neighbours ($kK$-NN) algorithm that explores the local curvature at a sample to adaptively defining the neighborhood size. Results on many real-world datasets indicate that the new $kK$-NN algorithm yields superior balanced accuracy compared to the established $k$-NN method.
  arXiv  Detail & Related papers  (2024-09-08T13:08:45Z)
• How to Fix a Broken Confidence Estimator: Evaluating Post-hoc Methods for Selective Classification with Deep Neural Networks [1.4502611532302039]
  We show that a simple $p$-norm normalization of the logits, followed by taking the maximum logit as the confidence estimator, can lead to considerable gains in selective classification performance. Our results are shown to be consistent under distribution shift.
  arXiv  Detail & Related papers  (2023-05-24T18:56:55Z)
• Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty [3.4208659698673127]
  Only increasing accuracy without considering uncertainty may negatively impact Deep Neural Network (DNN) decision-making. This paper proposes five combined preprocessing and post-processing methods for time-series binary classification problems.
  arXiv  Detail & Related papers  (2022-11-05T15:04:45Z)
• Optimizing Partial Area Under the Top-k Curve: Theory and Practice [151.5072746015253]
  We develop a novel metric named partial Area Under the top-k Curve (AUTKC) AUTKC has a better discrimination ability, and its Bayes optimal score function could give a correct top-K ranking with respect to the conditional probability. We present an empirical surrogate risk minimization framework to optimize the proposed metric.
  arXiv  Detail & Related papers  (2022-09-03T11:09:13Z)
• Sample-dependent Adaptive Temperature Scaling for Improved Calibration [95.7477042886242]
  Post-hoc approach to compensate for neural networks being wrong is to perform temperature scaling. We propose to predict a different temperature value for each input, allowing us to adjust the mismatch between confidence and accuracy. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-13T14:13:49Z)
• Test-time Batch Statistics Calibration for Covariate Shift [66.7044675981449]
  We propose to adapt the deep models to the novel environment during inference. We present a general formulation $alpha$-BN to calibrate the batch statistics. We also present a novel loss function to form a unified test time adaptation framework Core.
  arXiv  Detail & Related papers  (2021-10-06T08:45:03Z)
• Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection [85.53263670166304]
  One-stage detector basically formulates object detection as dense classification and localization. Recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization.
  arXiv  Detail & Related papers  (2020-06-08T07:24:33Z)
• Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations [154.51144248210338]
  Batch Nuclear-norm Maximization (BNM) is proposed to boost the learning under label insufficient learning scenarios. BNM outperforms competitors and works well with existing well-known methods.
  arXiv  Detail & Related papers  (2020-03-27T05:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.