Related papers: ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold

ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold

URL: http://arxiv.org/abs/2604.13392v1
Date: Wed, 15 Apr 2026 01:43:00 GMT
Title: ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold
Abstract summary: Tabular data is prevalent in high-stakes domains such as healthcare and finance.<n>While symbolic models offer verifiable logic, they lack semantic expressiveness.<n>We propose ReSS, a systematic framework that bridges symbolic and neural reasoning models.
Score: 38.12595356036359
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Tabular data remains prevalent in high-stakes domains such as healthcare and finance, where predictive models are expected to provide both high accuracy and faithful, human-understandable reasoning. While symbolic models offer verifiable logic, they lack semantic expressiveness. Meanwhile, general-purpose LLMs often require specialized fine-tuning to master domain-specific tabular reasoning. To address the dual challenges of scalable data curation and reasoning consistency, we propose ReSS, a systematic framework that bridges symbolic and neural reasoning models. ReSS leverages a decision-tree model to extract instance-level decision paths as symbolic scaffolds. These scaffolds, alongside input features and labels, guide an LLM to generate grounded natural-language reasoning that strictly adheres to the underlying decision logic. The resulting high-quality dataset is used to fine-tune a pretrained LLM into a specialized tabular reasoning model, further enhanced by a scaffold-invariant data augmentation strategy to improve generalization and explainability. To rigorously assess faithfulness, we introduce quantitative metrics including hallucination rate, explanation necessity, and explanation sufficiency. Experimental results on medical and financial benchmarks demonstrate that ReSS-trained models improve traditional decision trees and standard fine-tuning approaches up to $10\%$ while producing faithful and consistent reasoning

Related papers

Cognitive Profiling of LRMs' Reasoning Traces Using Bloom's Taxonomy [51.08110837378928]
We introduce a framework for automatic annotation of reasoning steps through the lens of Bloom's taxonomy.<n>We perform a large-scale analysis across models and datasets, revealing both similarities and differences in thinking patterns across models and tasks.<n>Our findings establish a fine-grained framework for analyzing thinking patterns in LRMs and provide actionable insights for enhancing reasoning quality.
arXiv Detail & Related papers (2026-08-24T12:55:51Z)
Enhancing LIME using Neural Decision Trees [0.906681787649105]
Local Interpretable Model-Agnostic Explanations (LIME) has been a very popular framework for interpretable machine learning.<n>We propose the NDT-LIME variant that integrates Neural Decision Trees (NDTs) as surrogate models.<n>By leveraging the structured, hierarchical nature of NDTs, our approach aims at providing more accurate and meaningful local explanations.
arXiv Detail & Related papers (2026-03-21T19:24:06Z)
Unlocking Reasoning Capability on Machine Translation in Large Language Models [57.60641851466707]
Reasoning-oriented large language models (RLMs) achieve strong gains on tasks such as mathematics and coding by generating explicit intermediate reasoning.<n>We systematically evaluate several open- and closed-weights RLMs on the WMT24++ benchmark.<n>We find that enabling explicit reasoning consistently degrades translation quality across languages and models.
arXiv Detail & Related papers (2026-02-16T14:05:59Z)
FORESTLLM: Large Language Models Make Random Forest Great on Few-shot Tabular Learning [20.27406245916013]
We propose a novel framework that unifies the structural inductive biases of decision forests with the semantic reasoning capabilities of large language models (LLMs)<n>Our method is two-fold. First, we introduce a semantic splitting criterion in which the LLM evaluates candidate partitions based on their coherence over both labeled and unlabeled data, enabling the induction of more robust and generalizable tree structures under few-shot supervision.<n>Second, we propose a one-time in-context inference mechanism for leaf node stabilization, where the LLM distills the decision path and its supporting examples into a concise, deterministic prediction, replacing noisy empirical estimates with semantically informed outputs
arXiv Detail & Related papers (2026-01-16T14:08:51Z)
How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns [51.02752099869218]
Large Language Models (LLMs) display strikingly different generalization behaviors.<n>We introduce a novel benchmark that decomposes reasoning into atomic core skills.<n>We show that RL-tuned models maintain more stable behavioral profiles and resist collapse in reasoning skills, whereas SFT models exhibit sharper drift and overfit to surface patterns.
arXiv Detail & Related papers (2025-12-30T08:16:20Z)
Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision [43.05159920832912]
Large language models (LLMs) have shown strong performance in many reasoning benchmarks.<n>LLMs are susceptible to content variations, demonstrating a lack of robust planning or symbolic abstractions.<n>We propose to overcome such limitations by synthesizing high-quality symbolic reasoning trajectories with stepwise pseudo-labels.
arXiv Detail & Related papers (2025-05-26T18:06:39Z)
Learning Decision Trees as Amortized Structure Inference [59.65621207449269]
We propose a hybrid amortized structure inference approach to learn predictive decision tree ensembles given data.<n>We show that our approach, DT-GFN, outperforms state-of-the-art decision tree and deep learning methods on standard classification benchmarks.
arXiv Detail & Related papers (2025-03-10T07:05:07Z)
Efficient or Powerful? Trade-offs Between Machine Learning and Deep Learning for Mental Illness Detection on Social Media [0.036136619420474754]
Social media platforms provide valuable insights into mental health trends by capturing user-generated discussions on conditions such as depression, anxiety, and suicidal ideation.<n>Machine learning (ML) and deep learning (DL) models have been increasingly applied to classify mental health conditions from textual data.<n>This study evaluates multiple ML models, including logistic regression, random forest, and LightGBM, alongside deep learning architectures such as ALBERT and Gated Recurrent Units (GRUs)<n>Our findings indicate that ML and DL models achieve comparable classification performance on medium-sized datasets.
arXiv Detail & Related papers (2025-03-03T00:51:41Z)
Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data [53.433309883370974]
This work explores the potential and limitations of using graph-based synthetic reasoning data as training signals to enhance Large Language Models' reasoning capabilities.<n>Our experiments, conducted on two established natural language reasoning tasks, demonstrate that supervised fine-tuning with synthetic graph-based reasoning data effectively enhances LLMs' reasoning performance without compromising their effectiveness on other standard evaluation benchmarks.
arXiv Detail & Related papers (2024-09-19T03:39:09Z)
Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference [17.636872632724582]
We introduce a neuro-symbolic natural logic framework based on reinforcement learning with introspective revision. The proposed model has built-in interpretability and shows superior capability in monotonicity inference, systematic generalization, and interpretability.
arXiv Detail & Related papers (2022-03-09T16:31:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.