STRABLE: Benchmarking Tabular Machine Learning with Strings

• URL: http://arxiv.org/abs/2605.12292v1
• Date: Tue, 12 May 2026 15:47:50 GMT
• Title: STRABLE: Benchmarking Tabular Machine Learning with Strings
• Abstract summary: STRABLE is a benchmarking corpus of 108 tables, all real-world learning problems with strings and numbers across diverse application fields.<n>We run the first large-scale empirical study of tabular learning with strings, evaluating 445 pipelines.<n>We find that, because most tables in the wild are categorical-dominant, advanced tabular learners paired with simple string embeddings achieve good predictions at low computational cost.
• Score: 53.03295517218137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Benchmarking tabular learning has revealed the benefit of dedicated architectures, pushing the state of the art. But real-world tables often contain string entries, beyond numbers, and these settings have been understudied due to a lack of a solid benchmarking suite. They lead to new research questions: Are dedicated learners needed, with end-to-end modeling of strings and numbers? Or does it suffice to encode strings as numbers, as with a categorical encoding? And if so, do the resulting tables resemble numerical tabular data, calling for the same learners? To enable these studies, we contribute STRABLE, a benchmarking corpus of 108 tables, all real-world learning problems with strings and numbers across diverse application fields. We run the first large-scale empirical study of tabular learning with strings, evaluating 445 pipelines. These pipelines span end-to-end architectures and modular pipelines, where strings are first encoded, then post-processed, and finally passed to a tabular learner. We find that, because most tables in the wild are categorical-dominant, advanced tabular learners paired with simple string embeddings achieve good predictions at low computational cost. On free-text-dominant tables, large LLM encoders become competitive. Their performance also appears sensitive to post-processing, with differences across LLM families. Finally, we show that STRABLE is a good set of tables to study "string tabular" learning as it leads to generalizable pipeline rankings that are close to the oracle rankings. We thus establish STRABLE as a foundation for research on tabular learning with strings, an important yet understudied area.

Related papers

• ProgramTab: Boosting Table Reasoning of LLMs via Programmatic Paradigm [35.68726822569786]
  Table-based reasoning with large language models (LLMs) requires reasoning based on natural language questions and structured data.<n>The text-to- mathematical approach is used to efficiently extract key information from tables and generate smaller sub-tables.<n>We propose the ProgramTab framework, which guides LLMs employing in-context learning to perform tabular data preprocessing with Python code.
  arXiv  Detail & Related papers  (2026-07-13T07:58:57Z)
• TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding [35.46103363192022]
  We introduce the Tabular Embedding Benchmark (TabBench) to evaluate the capability of embedding models.<n>We then propose TabEmbed, the first generalist embedding model that unifies tabular classification and retrieval within a shared embedding space.<n> Experimental results on TabBench demonstrate that TabEmbed significantly outperforms state-of-the-art text embedding models.
  arXiv  Detail & Related papers  (2026-05-06T14:22:34Z)
• TabGemma: Text-Based Tabular ICL via LLM using Continued Pretraining and Retrieval [1.4995278030955144]
  We introduce TabGemma, a schema-agnostic in-context learner that treats rows as sequences.<n>For inference, we use a compact n-gram-based retrieval to select informative exemplars that fit within a 128k-token window.<n>For regression, it is competitive at small sample sizes but trails conventional approaches as data grows.
  arXiv  Detail & Related papers  (2025-11-05T15:51:03Z)
• RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis [16.572608600078922]
  RealHiTBench is a benchmark designed to evaluate the performance of Large Language Models (LLMs) across a variety of input formats.<n>Our experimental results, using 25 state-of-the-art LLMs, demonstrate that RealHiTBench is indeed a challenging benchmark.<n>We also develop TreeThinker, a tree-based pipeline that organizes hierarchical headers into a tree structure.
  arXiv  Detail & Related papers  (2025-06-16T12:19:08Z)
• Tabular Transfer Learning via Prompting LLMs [52.96022335067357]
  We propose a novel framework, Prompt to Transfer (P2T), that utilizes unlabeled (or heterogeneous) source data with large language models (LLMs) P2T identifies a column feature in a source dataset that is strongly correlated with a target task feature to create examples relevant to the target task, thus creating pseudo-demonstrations for prompts.
  arXiv  Detail & Related papers  (2024-08-09T11:30:52Z)
• TabuLa: Harnessing Language Models for Tabular Data Synthesis [4.539846270369207]
  Tabula is a tabular data synthesizer that leverages the structure of large language models (LLMs)<n>Unlike state-of-the-art (SOTA) LLMs, Tabula discards the pre-trained weights originally designed for natural language tasks.<n> experiments show that Tabula achieves superior synthetic data utility compared to current SOTA methods.
  arXiv  Detail & Related papers  (2023-10-19T13:50:56Z)
• Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study [44.39031420687302]
  Large language models (LLMs) are becoming attractive as few-shot reasoners to solve Natural Language (NL)-related tasks. We try to understand this by designing a benchmark to evaluate the structural understanding capabilities of LLMs. We propose $textitself-augmentation$ for effective structural prompting, such as critical value / range identification.
  arXiv  Detail & Related papers  (2023-05-22T14:23:46Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [58.617025733655005]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning)<n>It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario.<n>Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tables [64.0903766169603]
  We propose a framework for few-shot semi-supervised learning, coined Self-generated Tasks from UNlabeled Tables (STUNT) Our key idea is to self-generate diverse few-shot tasks by treating randomly chosen columns as a target label. We then employ a meta-learning scheme to learn generalizable knowledge with the constructed tasks.
  arXiv  Detail & Related papers  (2023-03-02T02:37:54Z)
• TabLLM: Few-shot Classification of Tabular Data with Large Language Models [66.03023402174138]
  We study the application of large language models to zero-shot and few-shot classification. We evaluate several serialization methods including templates, table-to-text models, and large language models. This approach is also competitive with strong traditional baselines like gradient-boosted trees.
  arXiv  Detail & Related papers  (2022-10-19T17:08:13Z)
• Benchmarking Multimodal AutoML for Tabular Data with Text Fields [83.43249184357053]
  We assemble 18 multimodal data tables that each contain some text fields. Our benchmark enables researchers to evaluate their own methods for supervised learning with numeric, categorical, and text features.
  arXiv  Detail & Related papers  (2021-11-04T09:29:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.