Effects of structure on reasoning in instance-level Self-Discover

• URL: http://arxiv.org/abs/2507.03347v1
• Date: Fri, 04 Jul 2025 07:28:42 GMT
• Title: Effects of structure on reasoning in instance-level Self-Discover
• Authors: Sachith Gunasekara, Yasiru Ratnayake,
• Abstract summary: This paper introduces iSelf-Discover, an instance-level adaptation of the Self-Discover framework, and using it compares dynamically generated structured reasoning with its unstructured counterpart.<n>Our empirical evaluation across diverse benchmarks using state-of-the-art open-source models supports a consistent advantage for unstructured reasoning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The drive for predictable LLM reasoning in their integration with compound systems has popularized structured outputs, yet concerns remain about performance trade-offs compared to unconstrained natural language. At the same time, training on unconstrained Chain of Thought (CoT) traces has brought about a new class of strong reasoning models that nevertheless present novel compute budget and faithfulness challenges. This paper introduces iSelf-Discover, an instance-level adaptation of the Self-Discover framework, and using it compares dynamically generated structured JSON reasoning with its unstructured counterpart. Our empirical evaluation across diverse benchmarks using state-of-the-art open-source models supports a consistent advantage for unstructured reasoning. Notably, on the complex MATH benchmark, unstructured plans achieved relative performance improvements of up to 18.90\% over structured approaches. Zero-shot unstructured iSelf-Discover variants are also shown to outperform their five-shot structured counterparts, underscoring the significance of this gap, even when structured plans are dynamically generated to ensure reasoning precedes the final answer. We further demonstrate that the optimal granularity of plan generation (instance-level vs. task-level) is context-dependent. These findings invite re-evaluation of the reliance on structured formats for complex problem-solving and how compound systems should be organized.

Related papers

• Enhancing Large Language Models through Structured Reasoning [15.472375478049823]
  We introduce a novel approach to enhance Large Language Models (LLMs) through explicit structured reasoning.<n>First, we convert unstructured data into structured formats by explicitly annotating reasoning steps.<n>We then employ this structured dataset to train LLMs through Supervised Fine-Tuning (SFT)
  arXiv  Detail & Related papers  (2025-06-25T08:36:12Z)
• Modeling and Visualization Reasoning for Stakeholders in Education and Industry Integration Systems: Research on Structured Synthetic Dialogue Data Generation Based on NIST Standards [3.5516803380598074]
  This study addresses the structural complexity and semantic ambiguity in stakeholder interactions within the Education-Industry Integration (EII) system.<n>We propose a structural modeling paradigm based on the National Institute of Standards and Technology (NIST) synthetic data quality framework.
  arXiv  Detail & Related papers  (2025-06-20T12:37:43Z)
• AlphaFold Database Debiasing for Robust Inverse Folding [58.792020809180336]
  We introduce a Debiasing Structure AutoEncoder (DeSAE) that learns to reconstruct native-like conformations from intentionally corrupted backbone geometries.<n>At inference, applying DeSAE to AFDB structures produces debiased structures that significantly improve inverse folding performance.
  arXiv  Detail & Related papers  (2025-06-10T02:25:31Z)
• Tuning for Trustworthiness -- Balancing Performance and Explanation Consistency in Neural Network Optimization [49.567092222782435]
  We introduce the novel concept of XAI consistency, defined as the agreement among different feature attribution methods.<n>We create a multi-objective optimization framework that balances predictive performance with explanation.<n>Our research provides a foundation for future investigations into whether models from the trade-off zone-balancing performance loss and XAI consistency-exhibit greater robustness.
  arXiv  Detail & Related papers  (2025-05-12T13:19:14Z)
• Hierarchical Contextual Manifold Alignment for Structuring Latent Representations in Large Language Models [7.798982346197703]
  The organization of latent token representations plays a crucial role in determining the stability, generalization, and contextual consistency of language models.<n>A hierarchical alignment method was introduced to token embeddings without altering core model weights.<n> Experimental evaluations demonstrated improvements in rare token retrieval, adversarial, and long-range dependency tracking.
  arXiv  Detail & Related papers  (2025-02-06T04:01:27Z)
• Neural Contextual Reinforcement Framework for Logical Structure Language Generation [1.08272575635683]
  The framework integrates custom reward functions and dynamic context alignment mechanisms.<n>It produces outputs that align closely with human expectations of logical structure and semantic flow.<n>It exhibits robustness in handling noisy input data and scalability across varying model sizes.
  arXiv  Detail & Related papers  (2025-01-20T11:34:28Z)
• StructTest: Benchmarking LLMs' Reasoning through Compositional Structured Outputs [78.84060166851805]
  StructTest is a novel benchmark that evaluates large language models (LLMs) on their ability to follow compositional instructions and generate structured outputs.<n> Assessments are conducted deterministically using a rule-based evaluator, which can be easily extended to new tasks and datasets.<n>We demonstrate that StructTest remains challenging even for top-performing models like Deepseek-V3/R1 and GPT-4o.
  arXiv  Detail & Related papers  (2024-12-23T22:08:40Z)
• Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic [51.967603572656266]
  We introduce a consistent and theoretically grounded approach to annotating decompositional entailment. We find that our new dataset, RDTE, has a substantially higher internal consistency (+9%) than prior decompositional entailment datasets. We also find that training an RDTE-oriented entailment classifier via knowledge distillation and employing it in an entailment tree reasoning engine significantly improves both accuracy and proof quality.
  arXiv  Detail & Related papers  (2024-02-22T18:55:17Z)
• StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure [5.2869308707704255]
  StrAE is a Structured Autoencoder framework that through strict adherence to explicit structure, enables effective learning of multi-level representations. We show that our results are directly attributable to the informativeness of the structure provided as input, and show that this is not the case for existing tree models. We then extend StrAE to allow the model to define its own compositions using a simple localised-merge algorithm.
  arXiv  Detail & Related papers  (2023-05-09T16:20:48Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• Autoregressive Structured Prediction with Language Models [73.11519625765301]
  We describe an approach to model structures as sequences of actions in an autoregressive manner with PLMs. Our approach achieves the new state-of-the-art on all the structured prediction tasks we looked at.
  arXiv  Detail & Related papers  (2022-10-26T13:27:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.