TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding

• URL: http://arxiv.org/abs/2605.04962v1
• Date: Wed, 06 May 2026 14:22:34 GMT
• Title: TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding
• Abstract summary: We introduce the Tabular Embedding Benchmark (TabBench) to evaluate the capability of embedding models.<n>We then propose TabEmbed, the first generalist embedding model that unifies tabular classification and retrieval within a shared embedding space.<n> Experimental results on TabBench demonstrate that TabEmbed significantly outperforms state-of-the-art text embedding models.
• Score: 35.46103363192022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models have established unified representations for natural language processing, yet this paradigm remains largely unexplored for tabular data. Existing methods face fundamental limitations: LLM-based approaches lack retrieval-compatible vector outputs, whereas text embedding models often fail to capture tabular structure and numerical semantics. To bridge this gap, we first introduce the Tabular Embedding Benchmark (TabBench), a comprehensive suite designed to evaluate the tabular understanding capability of embedding models. We then propose TabEmbed, the first generalist embedding model that unifies tabular classification and retrieval within a shared embedding space. By reformulating diverse tabular tasks as semantic matching problems, TabEmbed leverages large-scale contrastive learning with positive-aware hard negative mining to discern fine-grained structural and numerical nuances. Experimental results on TabBench demonstrate that TabEmbed significantly outperforms state-of-the-art text embedding models, establishing a new baseline for universal tabular representation learning. Code and datasets are publicly available at https://github.com/qiangminjie27/TabEmbed and https://huggingface.co/datasets/qiangminjie27/TabBench.

Related papers

• SOMTab: Set-Order Mamba for Efficient Tabular In-Context Learning [30.163512246197843]
  We introduce SOMTab, a Set-Order Mamba architecture for efficient in-context learning.<n>For row and column representations, it maps unordered table tokens into stable latent slots and applies Mamba-based state-space mixing.<n>For final prediction, it retains attention-based in-context learning to preserve query-conditioned retrieval from labeled context examples.
  arXiv  Detail & Related papers  (2026-08-28T03:43:08Z)
• TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention [47.480923979431886]
  Tabular foundation models perform prediction via in-context learning, inferring test labels directly from labeled training examples.<n>Recent models often improve accuracy with increasingly complex architectures, incurring higher inference cost and limiting practical deployment.<n>We show that a lightweight row-wise attention-only backbone can remain highly competitive with two simple enhancements.
  arXiv  Detail & Related papers  (2026-06-05T14:55:30Z)
• Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks [15.117602507724726]
  Tabular representation learning models aim to learn universal representations of data that transfer across tasks and domains.<n>Existing methods are often evaluated under task-specific settings that make direct comparison difficult.<n>We introduce TEmBed, a benchmark for systematically evaluating embeddings across four representation levels.
  arXiv  Detail & Related papers  (2026-04-23T14:05:38Z)
• Comparing Task-Agnostic Embedding Models for Tabular Data [1.6479389738270018]
  This work specifically focuses on representation learning, i.e., on transferable, task-agnostic embeddings.<n>Tableizer features achieve comparable or superior performance while being up to three orders of magnitude faster than recent foundation models.
  arXiv  Detail & Related papers  (2025-11-18T09:10:40Z)
• Towards Benchmarking Foundation Models for Tabular Data With Text [36.3195231571412]
  We propose a series of simple yet effective ablation-style strategies for incorporating text into conventional pipelines.<n>We benchmark how state-of-the-art tabular foundation models can handle textual data by manually curating a collection of real-world datasets with meaningful textual features.
  arXiv  Detail & Related papers  (2025-07-10T15:01:31Z)
• ConTextTab: A Semantics-Aware Tabular In-Context Learner [1.0999592665107414]
  We introduce ConTextTab, integrating semantic understanding and alignment into a table-native ICL framework.<n>Our model is competitive with SOTA across a broad set of benchmarks while setting a new standard on the semantically rich CARTE benchmark.
  arXiv  Detail & Related papers  (2025-06-12T13:57:29Z)
• UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition [55.153629718464565]
  We introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure.
  arXiv  Detail & Related papers  (2024-09-20T01:26:32Z)
• LaTable: Towards Large Tabular Models [63.995130144110156]
  Tabular generative foundation models are hard to build due to the heterogeneous feature spaces of different datasets. LaTable is a novel diffusion model that addresses these challenges and can be trained across different datasets. We find that LaTable outperforms baselines on in-distribution generation, and that finetuning LaTable can generate out-of-distribution datasets better with fewer samples.
  arXiv  Detail & Related papers  (2024-06-25T16:03:50Z)
• Rethinking Pre-Training in Tabular Data: A Neighborhood Embedding Perspective [71.45945607871715]
  We propose Tabular data Pre-Training via Meta-representation (TabPTM)<n>The core idea is to embed data instances into a shared feature space, where each instance is represented by its distance to a fixed number of nearest neighbors and their labels.<n>Extensive experiments on 101 datasets confirm TabPTM's effectiveness in both classification and regression tasks, with and without fine-tuning.
  arXiv  Detail & Related papers  (2023-10-31T18:03:54Z)
• STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tables [64.0903766169603]
  We propose a framework for few-shot semi-supervised learning, coined Self-generated Tasks from UNlabeled Tables (STUNT) Our key idea is to self-generate diverse few-shot tasks by treating randomly chosen columns as a target label. We then employ a meta-learning scheme to learn generalizable knowledge with the constructed tasks.
  arXiv  Detail & Related papers  (2023-03-02T02:37:54Z)
• TabLLM: Few-shot Classification of Tabular Data with Large Language Models [66.03023402174138]
  We study the application of large language models to zero-shot and few-shot classification. We evaluate several serialization methods including templates, table-to-text models, and large language models. This approach is also competitive with strong traditional baselines like gradient-boosted trees.
  arXiv  Detail & Related papers  (2022-10-19T17:08:13Z)
• PTab: Using the Pre-trained Language Model for Modeling Tabular Data [5.791972449406902]
  Recent studies show that neural-based models are effective in learning contextual representation for Tabular data. We propose a novel framework PTab, using the Pre-trained language model to model Tabular data. Our method has achieved a better average AUC score in supervised settings compared to the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-09-15T08:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.