Graph-of-Causal Evolution: Challenging Chain-of-Model for Reasoning

• URL: http://arxiv.org/abs/2506.07501v1
• Date: Mon, 09 Jun 2025 07:26:47 GMT
• Title: Graph-of-Causal Evolution: Challenging Chain-of-Model for Reasoning
• Abstract summary: This work proposes a graph of causal evolution (GoCE)<n>GoCE maps the implicit token representation into a differentiable and sparse causal adjacency matrix, then permeate causal constraints through each layer of calculation.<n>It is evaluated on publicly available datasets including CLUTRR, CLADDER, EX-FEVER, and CausalQA.
• Score: 4.586907225774023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In view of the problem that each subchain in the chain-of-model (CoM) relies only on the information of the previous subchain and may lose long-range dependencies due to the causal mask blocking the global context flow between multi-level subchains, this work proposes a graph of causal evolution (GoCE). Its core principle is to map the implicit token representation into a differentiable and sparse causal adjacency matrix, then permeate causal constraints through each layer of calculation using causal-masked attention and causal-MoE. By combining intervention consistency loss test and self-evolution gate, the dynamic balance between causal structure learning and adaptive updating of transformer architecture is realized. The researcher built experimental environments in sandboxes built with Claude Sonnet 4, o4-mini-high, and DeepSeek R1 respectively with the transformer variant architecture introduced in GoCE. It is evaluated on publicly available datasets including CLUTRR, CLADDER, EX-FEVER, and CausalQA and compared with the baseline LLMs. The finding proves that GoCE strengthens the transformer's ability to capture long-range causal dependencies, while the ability to self-evolve is improved. It not only surpasses the design of CoM in terms of design principles, but also provides experience for future research on causal learning and continuous adaptive improvement.

Related papers

• How Developers Use Relation Chains in Gerrit-Based Review Ecosystems: An Empirical Study Across Three Open-Source Ecosystems [2.747398258852965]
  Developers increasingly coordinate dependent review by submitting sequences of related changes rather than monolithic ones.<n>In Gerrit, these dependencies form relation chains: structured review units that link changes together.<n>We analyze 29,580 relation chains from 15 repositories across three Gerrit ecosystems.
  arXiv  Detail & Related papers  (2026-07-22T14:15:39Z)
• INFUSER: Influence-Guided Self-Evolution Improves Reasoning [54.101135873140066]
  We introduce an iterative co-training framework with two co-evolving roles.<n>A solver is trained with standard correctness rewards against the generator-provided answers.<n>An 8B INF co-evolving generator outperforms a frozen 32B thinking generator on math and coding.
  arXiv  Detail & Related papers  (2026-06-08T05:40:36Z)
• Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models [39.290072292743226]
  We introduce Mix-GRM, a framework that reconfigures raw rationales into structured B-CoT and D-CoT through a modular synthesis pipeline.<n>Experiments demonstrate that Mix-GRM establishes a new state-of-the-art across five benchmarks, surpassing leading open-source RMs by an average of 8.2%.
  arXiv  Detail & Related papers  (2026-03-02T07:54:29Z)
• Learning Causality for Longitudinal Data [1.2691047660244335]
  This thesis develops methods for causal inference and causal representation learning in high-dimensional, time-varying data.<n>The first contribution introduces the Causal Dynamic Variational Autoencoder (CDVAE), a model for estimating Individual Treatment Effects (ITEs)<n>The second contribution proposes an efficient framework for long-term counterfactual regression based on RNNs enhanced with Contrastive Predictive Coding ( CPC) and InfoMax.<n>The third contribution advances CRL by addressing how latent causes manifest in observed variables.
  arXiv  Detail & Related papers  (2025-12-04T16:51:49Z)
• Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information [91.66597637613263]
  transformer-based models leveraging the attention mechanism have demonstrated strong empirical success in capturing complex dependencies within graphs.<n>We introduce a novel information-theoretic metric: the kernel-guided mutual information (KG-MI) based on the $f$-divergence.<n>We prove that, given sequences generated by a $K$-parent DAG, training a single-layer, multi-head transformer via a gradient ascent converges to the global optimum time.
  arXiv  Detail & Related papers  (2025-10-29T14:07:12Z)
• CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning [93.05917922306196]
  Composed Image Retrieval (CIR) aims to find a target image from a reference image and a modification text.<n>CIR-CoT is the first end-to-end retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2025-10-09T09:41:45Z)
• Structural Entropy Guided Probabilistic Coding [52.01765333755793]
  We propose a novel structural entropy-guided probabilistic coding model, named SEPC.<n>We incorporate the relationship between latent variables into the optimization by proposing a structural entropy regularization loss.<n> Experimental results across 12 natural language understanding tasks, including both classification and regression tasks, demonstrate the superior performance of SEPC.
  arXiv  Detail & Related papers  (2024-12-12T00:37:53Z)
• Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers [54.20763128054692]
  We study how a two-attention-layer transformer is trained to perform ICL on $n$-gram Markov chain data. We prove that the gradient flow with respect to a cross-entropy ICL loss converges to a limiting model.
  arXiv  Detail & Related papers  (2024-09-09T18:10:26Z)
• Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs [63.36637269634553]
  We introduce a novel approach where LLMs are fine-tuned to generate a sequence of Diverse Chains of Thought (DCoT) within a single inference step.<n>We show that fine-tuning on DCoT improves performance over the CoT baseline across model families and scales.<n>Our work is also significant because both quantitative analyses and manual evaluations reveal the observed gains stem from the models' ability to refine an initial reasoning chain.
  arXiv  Detail & Related papers  (2024-07-03T15:01:18Z)
• ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting [124.69672273754144]
  Chain-of-Thought (CoT) prompting can enhance the reasoning capabilities of large language models (LLMs) Existing CoT approaches usually focus on simpler reasoning tasks and thus result in low-quality and inconsistent CoT prompts. We introduce CoTGenius, a novel framework designed for the automatic generation of superior CoT prompts.
  arXiv  Detail & Related papers  (2024-03-21T11:34:26Z)
• How Transformers Learn Causal Structure with Gradient Descent [44.31729147722701]
  Self-attention allows transformers to encode causal structure. We introduce an in-context learning task that requires learning latent causal structure. We show that transformers trained on our in-context learning task are able to recover a wide variety of causal structures.
  arXiv  Detail & Related papers  (2024-02-22T17:47:03Z)
• Causality is all you need [63.10680366545293]
  Causal Graph Routing (CGR) is an integrated causal scheme relying entirely on the intervention mechanisms to reveal the cause-effect forces hidden in data. CGR can surpass the current state-of-the-art methods on both Visual Question Answer and Long Document Classification tasks.
  arXiv  Detail & Related papers  (2023-11-21T02:53:40Z)
• Concept-free Causal Disentanglement with Variational Graph Auto-Encoder [4.171080633895958]
  We propose an unsupervised solution, dubbed concept-free causal disentanglement, built on a theoretically provable tight upper bound approximating the optimal factor. Based on this idea, we propose Concept-free Causal VGAE (CCVGAE) by incorporating a novel causal disentanglement layer into Variational Graph Auto-Encoder. We prove concept consistency under our concept-free causal disentanglement framework, hence employing it to enhance the meta-learning framework, called concept-free causal Meta-Graph.
  arXiv  Detail & Related papers  (2023-11-17T16:50:00Z)
• Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL [86.0987896274354]
  We first identify a fundamental pattern, self-excitation, as the primary cause of Q-value estimation divergence in offline RL. We then propose a novel Self-Excite Eigenvalue Measure (SEEM) metric to measure the evolving property of Q-network at training. For the first time, our theory can reliably decide whether the training will diverge at an early stage.
  arXiv  Detail & Related papers  (2023-10-06T17:57:44Z)
• Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detection [13.801572236048601]
  FOcus-the-Discrepancy (FOD) can simultaneously spot the patch-wise, intra- and inter-discrepancies of anomalies. In this paper, we propose a novel AD framework: FOcus-the-Discrepancy (FOD), which can simultaneously spot the patch-wise, intra- and inter-discrepancies of anomalies.
  arXiv  Detail & Related papers  (2023-08-06T01:30:26Z)
• End-to-End Supervised Multilabel Contrastive Learning [38.26579519598804]
  Multilabel representation learning is recognized as a challenging problem that can be associated with either label dependencies between object categories or data-related issues. Recent advances address these challenges from model- and data-centric viewpoints. We propose a new end-to-end training framework -- dubbed KMCL -- to address the shortcomings of both model- and data-centric designs.
  arXiv  Detail & Related papers  (2023-07-08T12:46:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.