Localized TabICLv2: Scaling Tabular In-Context Learning through k-NN

• URL: http://arxiv.org/abs/2608.16429v1
• Date: Mon, 17 Aug 2026 11:30:56 GMT
• Title: Localized TabICLv2: Scaling Tabular In-Context Learning through k-NN
• Abstract summary: Localized TabICLv2 reduces the inference cost of TabICLv2 by retrieving only the k nearest training neighbours for each test point.<n>On TabArena classification tasks, the fine-tuned localized model retains 98.64% of Full TabICLv2 accuracy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundational models for tabular data have made significant progress in recent years, with TabICLv2 reporting state-of-the-art performance on several tabular classification tasks. However, full-context tabular ICL still suffers from attention cost that grows with the training-context size, which limits its ability to handle large datasets efficiently. Localized TabICLv2 introduces a method that reduces the inference cost of TabICLv2 by retrieving only the k nearest training neighbours for each test point, measured by similarity in the model's Stage 2 row-representation space, rather than using the full training context. This requires no architectural changes, and we show that accuracy retention can be improved through additional Stage 2 and Stage 3 fine-tuning. On TabArena classification tasks, the fine-tuned localized model retains 98.64% of Full TabICLv2 accuracy and it achieves a median 2.18$\times$ speedup in batch inference, and reaches approximately 249$\times$ median speedup in the single-query serving setting.

Related papers

• TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention [47.480923979431886]
  Tabular foundation models perform prediction via in-context learning, inferring test labels directly from labeled training examples.<n>Recent models often improve accuracy with increasingly complex architectures, incurring higher inference cost and limiting practical deployment.<n>We show that a lightweight row-wise attention-only backbone can remain highly competitive with two simple enhancements.
  arXiv  Detail & Related papers  (2026-06-05T14:55:30Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• TabH2O: A Unified Foundation Model for Tabular Prediction [12.882141870947706]
  We present TabH2O, a model that performs classification and regression in a single forward pass via in-context learning.<n>We evaluate TabH2O v1 (29.2M parameters) on the TALENT benchmark, where it achieves an average rank of 2.55 out of 6 evaluated methods.
  arXiv  Detail & Related papers  (2026-05-18T13:27:54Z)
• TabICLv2: A better, faster, scalable, and open tabular foundation model [18.594859017648346]
  We introduce TabICLv2, a new state-of-the-art foundation model for regression and classification built on three pillars.<n>Tabiclv2 generalizes effectively to million-scale datasets under 50GB GPU memory while being markedly faster than RealTabPFN-2.5.
  arXiv  Detail & Related papers  (2026-02-11T18:51:02Z)
• Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training [53.07879717463279]
  textscDomain2Vec decomposes any dataset into a linear combination of several emphmeta-domains<n>textscDomain2Vec helps find the data mixture that enhances downstream task performance with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-06-12T17:53:51Z)
• A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities [51.08999772842298]
  Tabular Prior-data Fitted Network v2 (TabPFN v2) achieves unprecedented in-context learning performance across diverse downstream datasets.<n>We show that TabPFN v2 can infer attribute relationships even when provided with randomized attribute token inputs.<n>We demonstrate that TabPFN v2's limitations can be addressed through a test-time divide-and-context strategy.
  arXiv  Detail & Related papers  (2025-02-24T17:38:42Z)
• TabICL: A Tabular Foundation Model for In-Context Learning on Large Data [15.08819125687632]
  We introduce TabICL, a tabular foundation model for classification, pretrained on synthetic datasets with up to 60K samples.<n>Across 200 classification datasets from the TALENT benchmark, TabICL is on par with TabPFNv2 while being systematically faster (up to 10 times)<n>On 53 datasets with over 10K samples, TabICL surpasses both TabPFNv2 and CatBoost, demonstrating the potential of ICL for large data.
  arXiv  Detail & Related papers  (2025-02-08T13:25:04Z)
• TabDPT: Scaling Tabular Foundation Models on Real Data [20.00390825519329]
  We propose an approach to combine ICL-based retrieval with self supervised learning to train foundation models.<n>We show that incorporating real data during the pre-training phase can lead to significantly faster training and better generalization to unseen data.<n>Our resulting model, TabDPT, achieves top performance on both regression (CTR23) and classification (CC18) benchmarks.
  arXiv  Detail & Related papers  (2024-10-23T18:00:00Z)
• Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation [123.4883806344334]
  We study a realistic Continual Learning setting where learning algorithms are granted a restricted computational budget per time step while training. We apply this setting to large-scale semi-supervised Continual Learning scenarios with sparse label rates. Our extensive analysis and ablations demonstrate that DietCL is stable under a full spectrum of label sparsity, computational budget, and various other ablations.
  arXiv  Detail & Related papers  (2024-04-19T10:10:39Z)
• Improving Calibration for Long-Tailed Recognition [68.32848696795519]
  We propose two methods to improve calibration and performance in such scenarios. For dataset bias due to different samplers, we propose shifted batch normalization. Our proposed methods set new records on multiple popular long-tailed recognition benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-01T13:55:21Z)
• Understanding tables with intermediate pre-training [11.96734018295146]
  We adapt TAPAS, a table-based BERT model, to recognize entailment. We evaluate table pruning techniques as a pre-processing step to drastically improve the training and prediction efficiency.
  arXiv  Detail & Related papers  (2020-10-01T17:43:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.