FollowTable: A Benchmark for Instruction-Following Table Retrieval

• URL: http://arxiv.org/abs/2605.00400v1
• Date: Fri, 01 May 2026 04:42:13 GMT
• Title: FollowTable: A Benchmark for Instruction-Following Table Retrieval
• Abstract summary: We formalize Instruction-Following Table Retrieval (IFTR), a new task that requires models to jointly satisfy topical relevance and fine-grained instruction constraints.<n>We introduce FollowTable, the first large-scale benchmark for IFTR, constructed via a taxonomy-driven annotation pipeline.
• Score: 30.435522584995336
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Table Retrieval (TR) has traditionally been formulated as an ad-hoc retrieval problem, where relevance is primarily determined by topical semantic similarity. With the growing adoption of LLM-based agentic systems, access to structured data is increasingly instruction-driven, where relevance is conditional on explicit content and schema constraints rather than topical similarity alone. We therefore formalize Instruction-Following Table Retrieval (IFTR), a new task that requires models to jointly satisfy topical relevance and fine-grained instruction constraints. We identify two core challenges in IFTR: (i) sensitivity to content scope, such as inclusion and exclusion constraints, and (ii) awareness of schema-grounded requirements, including column semantics and representation granularity--capabilities largely absent in existing retrievers. To support systematic evaluation, we introduce FollowTable, the first large-scale benchmark for IFTR, constructed via a taxonomy-driven annotation pipeline. We further propose a new metric, termed the Instruction Responsiveness Score, to evaluate whether retrieval rankings consistently adapt to user instructions relative to a topic-only baseline. Our results indicate that existing retrieval models struggle to follow fine-grained instructions over tabular data. In particular, they exhibit systematic biases toward surface-level semantic cues and remain limited in handling schema-grounded constraints, highlighting substantial room for future improvements.

Related papers

• When Should Queries Be Decomposed? A Stage-Aware Study of Query Decomposition for Multi-Condition Retrieval [7.14438992840294]
  Multi-condition retrieval requires systems to identify documents that satisfy multiple distinct constraints.<n> query decomposition is widely adopted as an intuitive remedy, but its effectiveness across different retrieval pipeline stages remains underexplored.<n>We propose a principled Stage-Aware Decomposition framework that retains the monolithic query during initial retrieval to preserve global semantic context, while employing sub-queries exclusively during reranking for fine-grained constraint matching.
  arXiv  Detail & Related papers  (2026-06-07T11:13:15Z)
• TopBench: A Benchmark for Implicit Prediction and Reasoning over Tabular Question Answering [80.93487993878836]
  A common class of real-world queries is implicitly predictive, requiring the inference of unobserved answers from historical patterns rather than mere retrieval.<n>These queries introduce two challenges: recognizing latent intent and reliable predictive reasoning over massive tables.<n>We introduce TopBench, a benchmark consisting of 779 samples across four sub-tasks, ranging from single-point prediction to decision making, treatment effect analysis, and complex reasoning.
  arXiv  Detail & Related papers  (2026-04-30T16:22:51Z)
• ASTRA: Adaptive Semantic Tree Reasoning Architecture for Complex Table Question Answering [55.55968342644846]
  Table serialization remains a critical bottleneck for Large Language Models (LLMs) in complex table question answering.<n>Existing serialization methods fail to capture explicit hierarchies and lack schema flexibility.<n>We propose ASTRA (Adaptive Semantic Tree Reasoning Architecture) including two main modules, AdaSTR and DuTR.<n> Experiments on complex table benchmarks demonstrate that our method achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-04-10T06:09:41Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation [55.47971671635531]
  Large Language Models (LLMs) have shown remarkable performance on general Question Answering (QA)<n>Retrieval-Augmented Generation (RAG) addresses this limitation by enriching LLMs with external knowledge.<n>Existing systems primarily rely on unstructured documents, while largely overlooking relational databases.
  arXiv  Detail & Related papers  (2025-09-30T22:19:44Z)
• Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations [0.0]
  We propose a hierarchical clustering-based retrieval method that eliminates the need to predefine k.<n>Our approach maintains the accuracy and relevance of system responses while adaptively selecting semantically relevant content.<n>Our framework is simple to implement and easily integrates with existing RAG pipelines, making it a practical solution for real-world applications under limited resources.
  arXiv  Detail & Related papers  (2025-06-16T15:34:29Z)
• LLM-Symbolic Integration for Robust Temporal Tabular Reasoning [69.27153114778748]
  We introduce TempTabQA-C, a synthetic dataset designed for systematic and controlled evaluations.<n>This structured approach allows Large Language Models (LLMs) to generate and executesql queries, enhancing generalization and mitigating biases.
  arXiv  Detail & Related papers  (2025-06-06T05:14:04Z)
• Something's Fishy In The Data Lake: A Critical Re-evaluation of Table Union Search Benchmarks [0.9374652839580181]
  Recent table representation learning and data discovery methods tackle table union search (TUS) within data lakes.<n>These methods are commonly evaluated using benchmarks that aim to assess semantic understanding in real-world TUS tasks.<n>We propose essential criteria for future benchmarks to enable a more realistic and reliable evaluation of progress in semantic table union search.
  arXiv  Detail & Related papers  (2025-05-27T15:23:52Z)
• Reasoning-Aware Query-Focused Summarization over Multi-Table Data [1.325953054381901]
  We propose QueryTableSummarizer++, an end-to-end generative framework leveraging large language models (LLMs)<n>Our method eliminates the need for intermediate serialization steps and directly generates query-relevant summaries.<n> Experiments on a benchmark dataset demonstrate that QueryTableSummarizer++ significantly outperforms state-of-the-art baselines in terms of BLEU, ROUGE, and F1-score.
  arXiv  Detail & Related papers  (2024-12-12T06:04:31Z)
• Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion [50.03041373044267]
  We propose a Supervised Contrastive learning framework with adaptive classification criterion for Continual Learning. Experiments show that CFL achieves state-of-the-art performance and has a stronger ability to overcome compared with the classification baselines.
  arXiv  Detail & Related papers  (2023-05-20T19:22:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.