Causal Concept Graphs in LLM Latent Space for Stepwise Reasoning

• URL: http://arxiv.org/abs/2603.10377v1
• Date: Wed, 11 Mar 2026 03:46:38 GMT
• Title: Causal Concept Graphs in LLM Latent Space for Stepwise Reasoning
• Abstract summary: Causal Concept Graphs (CCG): a directed acyclic graph over sparse, interpretable latent features.<n>We combine task-conditioned sparse autoencoders for concept discovery with DAGMA-style differentiable structure learning for graph recovery.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse autoencoders can localize where concepts live in language models, but not how they interact during multi-step reasoning. We propose Causal Concept Graphs (CCG): a directed acyclic graph over sparse, interpretable latent features, where edges capture learned causal dependencies between concepts. We combine task-conditioned sparse autoencoders for concept discovery with DAGMA-style differentiable structure learning for graph recovery and introduce the Causal Fidelity Score (CFS) to evaluate whether graph-guided interventions induce larger downstream effects than random ones. On ARC-Challenge, StrategyQA, and LogiQA with GPT-2 Medium, across five seeds ($n{=}15$ paired runs), CCG achieves $\CFS=5.654\pm0.625$, outperforming ROME-style tracing ($3.382\pm0.233$), SAE-only ranking ($2.479\pm0.196$), and a random baseline ($1.032\pm0.034$), with $p<0.0001$ after Bonferroni correction. Learned graphs are sparse (5-6\% edge density), domain-specific, and stable across seeds.

Related papers

• TGHE: Template-based Graph Homomorphic Encryption for Privacy-Preserving GNN Inference in Edge-Cloud Systems [1.9552564628602804]
  Existing homomorphic encryption (HE)-based GNN systems adopt a graphcentric paradigm that couples per-query cost to global graph size.<n>We propose TGHE, an ego-centric framework that resolves this by exploiting a template phenomenon: local trees in transaction graphs converge into a small set of structural shapes.<n> TGHE canonicalizes ego-graphs at the edge and packs structurally identical trees into shared ciphertexts for SIMD-parallel encrypted inference.
  arXiv  Detail & Related papers  (2026-06-25T06:55:06Z)
• Multi-Modal Spatio-Temporal Graph Neural Network with Mixture of Experts for Soil Organic Carbon Prediction [0.0]
  Existing approaches face two limitations: they pair hand-crafted co-modals with classical ML or single-modal deep models that miss rich spectral and temporal information.<n>We introduce SpTG-NN, a multi-temporal graph neural network addressing both SpTGNN and grid-based architectures.<n>A fine-tuned TerraMind encoder extracts node features from Sentinel-2, Sentinel-1 and DEM signals.
  arXiv  Detail & Related papers  (2026-06-15T11:25:38Z)
• A Closed-Form Adaptive-Landmark Kernel for Certified Point-Cloud and Graph Classification [0.0]
  PALACE is the strongest closed-form diagram-based method on OrbitHS5k ($91.3pm 1.0%$, matching Persformer)<n>At $8$ domain inflation, placement maintains $94%$ while the uniform grid collapses to chance ($25%$ on 4-class data)<n> Empirically, PALACE is the strongest closed-form diagram-based method on OrbitHS5k ($91.3pm 1.0%$, matching Persformer)
  arXiv  Detail & Related papers  (2026-05-05T17:59:18Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• Scalable Attribute-Missing Graph Clustering via Neighborhood Differentiation [46.17999216122895]
  We propose a novel DGC method termed underlinetextbfComplementary underlinetextbfMulti-underlinetextbfView.<n> CMV-ND preprocesses graph structural information into multiple views in a complete but non-redundant manner.<n> Experimental results on six widely used graph datasets demonstrate that CMV-ND significantly improves the performance of various methods.
  arXiv  Detail & Related papers  (2025-07-09T13:42:43Z)
• Align-GRAG: Reasoning-Guided Dual Alignment for Graph Retrieval-Augmented Generation [79.75818239774952]
  Large language models (LLMs) have demonstrated remarkable capabilities, but still struggle with issues like hallucinations and outdated information.<n>Retrieval-augmented generation (RAG) addresses these issues by grounding LLM outputs in external knowledge with an Information Retrieval (IR) system.<n>We propose Align-GRAG, a novel reasoning-guided dual alignment framework in post-retrieval phrase.
  arXiv  Detail & Related papers  (2025-05-22T05:15:27Z)
• Adaptive Local Clustering over Attributed Graphs [16.197833892857282]
  Local graph clustering aims to identify a subgraph $C_s in G$ surrounding $v_s$ in time roughly linear with the size of $C_s$.<n>Most existing solutions merely rely on the topological connectivity between nodes in $G$, rendering them vulnerable to missing or noisy links.<n>We propose LACA, an efficient and effective approach for LGC that achieves superb empirical performance on multiple real datasets.
  arXiv  Detail & Related papers  (2025-03-26T12:24:07Z)
• Graph Sparsification via Mixture of Graphs [67.40204130771967]
  We introduce Mixture-of-Graphs (MoG) to dynamically select tailored pruning solutions for each node. MoG incorporates multiple sparsifier experts, each characterized by unique sparsity levels and pruning criteria, and selects the appropriate experts for each node. Experiments on four large-scale OGB datasets and two superpixel datasets, equipped with five GNNs, demonstrate that MoG identifies subgraphs at higher sparsity levels.
  arXiv  Detail & Related papers  (2024-05-23T07:40:21Z)
• Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness [80.87683145376305]
  Graph Neural Networks (GNNs) excel in various graph learning tasks but face computational challenges when applied to large-scale graphs. We propose Graph Sparse Training ( GST), which dynamically manipulates sparsity at the data level. GST produces a sparse graph with maximum topological integrity and no performance degradation.
  arXiv  Detail & Related papers  (2024-02-02T09:10:35Z)
• Collaborative likelihood-ratio estimation over graphs [55.98760097296213]
  Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF) We develop this idea in a concrete non-parametric method that we call Graph-based Relative Unconstrained Least-squares Importance Fitting (GRULSIF) We derive convergence rates for our collaborative approach that highlights the role played by variables such as the number of available observations per node, the size of the graph, and how accurately the graph structure encodes the similarity between tasks.
  arXiv  Detail & Related papers  (2022-05-28T15:37:03Z)
• Seeded graph matching for the correlated Gaussian Wigner model via the projected power method [5.639451539396459]
  We analyse the performance of the emphprojected power method (PPM) as a emphseeded graph matching algorithm. PPM works even in iterations of constant $sigma$, thus extending the analysis in (Mao et al. 2023) for the sparse Correlated Erdos-Renyi(CER) model to the (dense) CGW model.
  arXiv  Detail & Related papers  (2022-04-08T14:31:26Z)
• Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction [123.20238648121445]
  We propose a new self-supervised learning framework, Graph Information Aided Node feature exTraction (GIANT) GIANT makes use of the eXtreme Multi-label Classification (XMC) formalism, which is crucial for fine-tuning the language model based on graph information. We demonstrate the superior performance of GIANT over the standard GNN pipeline on Open Graph Benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-29T19:55:12Z)
• Adversarial Linear Contextual Bandits with Graph-Structured Side Observations [80.95090605985042]
  A learning agent repeatedly chooses from a set of $K$ actions after being presented with a $d$-dimensional context vector. The agent incurs and observes the loss of the chosen action, but also observes the losses of its neighboring actions in the observation structures. Two efficient algorithms are developed based on textttEXP3.
  arXiv  Detail & Related papers  (2020-12-10T15:40:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.