RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners

• URL: http://arxiv.org/abs/2605.00199v2
• Date: Thu, 07 May 2026 14:14:11 GMT
• Title: RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners
• Abstract summary: RSAT produces step-by-step reasoning cell-level citations grounded in table evidence.<n>Phase 1 teaches a structured output format from verified reasoning traces.<n>RSAT improves faithfulness 3.7$times$ over SFT alone (0.224$rightarrow$0.826), with near-perfect citation validity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: When a language model answers a table question, users have no way to verify which cells informed which reasoning steps. We introduce RSAT, a method that trains small language models (SLMs, 1-8B) to produce step-by-step reasoning with cell-level citations grounded in table evidence. Phase 1 (SFT) teaches a structured JSON output format from verified reasoning traces. Phase 2 (GRPO) optimizes a composite reward centered on NLI-based faithfulness, alongside citation validity and parsimony. Across six models from two families-Qwen 2.5 (1.5B/3B/7B) and Llama 3 (1B/3B/8B)-RSAT improves faithfulness 3.7$\times$ over SFT alone (0.224$\rightarrow$0.826), with near-perfect citation validity (0.992). Post-hoc attribution collapses below 13% format success, confirming that attribution must be integrated into reasoning, not retrofitted. Ablations show the faithfulness reward is essential: removing it drops faithfulness from 0.97 to 0.03.

Related papers

• Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection [66.44527094471619]
  We present RECAL, an unsupervised framework using relation-balanced masked graph learning to better capture rare interaction patterns.<n>On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset by 0.88, 0.82, and 0.42 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-09-15T00:31:01Z)
• State of Thought Enables Endogenous Reasoning [48.902117018115256]
  Test-time reasoning has emerged as a major approach to improving the capabilities of Large Language Models (LLMs)<n>We propose State of Thought, a new reasoning paradigm that enables endogenous reasoning in LLMs.<n>We show that SoT consistently improves mean-baseline accuracy while reducing generated tokens by 62.6% and end-to-end latency by 44.6%.
  arXiv  Detail & Related papers  (2026-09-13T05:55:03Z)
• Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators [3.0130431031064613]
  Hidden CoT Detection Score (HCDS) measures whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no-CoT.<n>HCDS does not directly observe or prove an unexposed reasoning trace.<n>Findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model.
  arXiv  Detail & Related papers  (2026-08-30T18:37:46Z)
• V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning [59.47823383593251]
  Vision-sampling models can produce fluent answers that are insufficiently grounded in the visual evidence.<n>We introduce Visuals-Based Reinforcement Learning, which decomposes reference responses into atomic propositions.<n>We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit.
  arXiv  Detail & Related papers  (2026-08-26T09:40:46Z)
• Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs [4.721990925113432]
  Aligned language models routinely misreport under non-evidential pressure.<n>We study the two demands, resist (ignore forbidden pressure) and update (follow licensed evidence)<n>We causally localize low-rank report coordinates for answer, confidence, and caveat.
  arXiv  Detail & Related papers  (2026-07-14T17:28:25Z)
• Statistically Indistinguishable, Operationally Distinct: A Formal Barrier for Tabular Foundation Models [6.240573495762223]
  We show that Tabular foundation models cannot reason about data produced by running systems without access to the rules that govern them.<n>In three matched $100$-state frontier large-lemma-model (LLM) runs, models given the schema, trigger source, rule tables, and state files classify at most $2/50$ legal states as Legal.<n>The barrier is identifiability, not capacity scale, data, and richer features cannot cross it without operational grounding.
  arXiv  Detail & Related papers  (2026-06-27T21:14:06Z)
• The Reasoning Trap: An Information-Theoretic Bound on Closed-System Multi-Step LLM Reasoning [0.0]
  We name the multi-agent case the Debate Trap and the broader phenomenon the Reasoning Trap.<n>The framework has three parts: (i) SFS (Supported Faithfulness Score), a claim-level metric verifying atomic claims against provided evidence; (ii) EGSR (Evidence-Grounded Socratic Reasoning), replacing adversarial argumentation with evidence-grounded inquiry.<n>We offer one falsifiable conjecture: any closed-system reasoning protocol preserving Theorem 1's Markov structure is, in expectation, subject to the same DPI bound.
  arXiv  Detail & Related papers  (2026-05-03T04:12:21Z)
• Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Models [0.0]
  LLMs deployed for natural-language querying of analytical databases suffer from two intertwined failures.<n>We benchmark three frontier LLMs on 100 natural-language questions over the Cleaned Contoso Retail dataset in ClickHouse.
  arXiv  Detail & Related papers  (2026-04-28T02:53:23Z)
• Learn by Surprise, Commit by Proof [0.0]
  We propose a self-gated post-training framework for autonomous knowledge acquisition.<n>When a passage produces anomalously high pertoken loss, L SCP flags it, generates a Q&A chain that forces the model to articulate its own knowledge.<n>The entire learning intensity is governed by a single parameter $r$.
  arXiv  Detail & Related papers  (2026-04-02T12:17:10Z)
• Epistemic Observability in Language Models [0.0]
  We find that models report highest confidence precisely when they are fabricating.<n>We prove, under formal assumptions, that this is not a capability gap but an observational one.<n>We construct a tensor interface that escapes the impossibility by exporting computational byproducts.
  arXiv  Detail & Related papers  (2026-03-20T21:59:34Z)
• How Does Prefix Matter in Reasoning Model Tuning? [57.69882799751655]
  We fine-tune three R1 series models across three core model capabilities: reasoning (mathematics), coding, safety, and factuality.<n>Results show that prefix-conditioned SFT improves both safety and reasoning performance, yielding up to +6% higher Safe@1 accuracy.
  arXiv  Detail & Related papers  (2026-01-04T18:04:23Z)
• ARCHE: A Novel Task to Evaluate LLMs on Latent Reasoning Chain Extraction [70.53044880892196]
  We introduce a novel task named Latent Reasoning Chain Extraction (ARCHE), in which models must decompose complex reasoning arguments into combinations of standard reasoning paradigms in the form of a Reasoning Logic Tree (RLT)<n>To facilitate this task, we release ARCHE Bench, a new benchmark derived from 70 Nature Communications articles, including more than 1,900 references and 38,000 viewpoints.<n> Evaluations on 10 leading LLMs on ARCHE Bench reveal that models exhibit a trade-off between REA and EC, and none are yet able to extract a complete and standard reasoning chain.
  arXiv  Detail & Related papers  (2025-11-16T07:37:09Z)
• Improving the fact-checking performance of language models by relying on their entailment ability [3.371541812350348]
  We propose a simple yet effective strategy to train encoder-only language models (ELMs) for fact-checking.<n>We conducted a rigorous set of experiments, comparing our approach with recent works and various prompting and fine-tuning strategies to demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2025-05-21T03:15:06Z)
• SIFT: Grounding LLM Reasoning in Contexts via Stickers [8.677892927579384]
  This paper identifies the misinterpretation of the context can be a significant issue during the reasoning process of large language models.<n>We introduce a novel, post-training approach called **Stick to the Facts (SIFT)** to tackle this.<n>At the core of SIFT lies the *Sticker*, which is generated by the model itself to explicitly emphasize the key information within the context.
  arXiv  Detail & Related papers  (2025-02-19T17:38:46Z)
• RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesis [84.57932472551889]
  RALL-E is a robust language modeling method for text-to-speech synthesis. RALL-E improves the WER of zero-shot TTS from $5.6%$ (without reranking) to $2.5%$ and $1.0%$, respectively.
  arXiv  Detail & Related papers  (2024-04-04T05:15:07Z)
• Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models [50.15455336684986]
  We evaluate the effectiveness of LogProbs and basic prompting to measure semantic plausibility. We find that LogProbs offers a more reliable measure of semantic plausibility than direct zero-shot prompting. We conclude that, even in the era of prompt-based evaluations, LogProbs constitute a useful metric of semantic plausibility.
  arXiv  Detail & Related papers  (2024-03-21T22:08:44Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.