Dynamics Reveals Structure: Challenging the Linear Propagation Assumption

• URL: http://arxiv.org/abs/2601.21601v1
• Date: Thu, 29 Jan 2026 12:08:00 GMT
• Title: Dynamics Reveals Structure: Challenging the Linear Propagation Assumption
• Authors: Hoyeon Chang, Bálint Mucsányi, Seong Joon Oh,
• Abstract summary: We study whether local updates coherently propagate to logical consequences.<n>For negation and converse, we prove that guaranteeing direction-agnostic first-order propagation requires a tensor factorization separating entity-pair context from relation content.<n>We show that composition reduces to conjunction, and prove that any conjunction well-defined on linear features must be bilinear.
• Score: 23.984006583336498
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks adapt through first-order parameter updates, yet it remains unclear whether such updates preserve logical coherence. We investigate the geometric limits of the Linear Propagation Assumption (LPA), the premise that local updates coherently propagate to logical consequences. To formalize this, we adopt relation algebra and study three core operations on relations: negation flips truth values, converse swaps argument order, and composition chains relations. For negation and converse, we prove that guaranteeing direction-agnostic first-order propagation necessitates a tensor factorization separating entity-pair context from relation content. However, for composition, we identify a fundamental obstruction. We show that composition reduces to conjunction, and prove that any conjunction well-defined on linear features must be bilinear. Since bilinearity is incompatible with negation, this forces the feature map to collapse. These results suggest that failures in knowledge editing, the reversal curse, and multi-hop reasoning may stem from common structural limitations inherent to the LPA.

Related papers

• CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching [50.65932158912512]
  We propose a new causal reasoning benchmark, CausalFlip, to encourage the development of new large language models.<n>CaulFlip consists of causal judgment questions built over event triples that could form different confounder, chain, and collider relations.<n>We evaluate LLMs under multiple training paradigms, including answer-only training, explicit Chain-of-Thought supervision, and a proposed internalized causal reasoning approach.
  arXiv  Detail & Related papers  (2026-02-23T18:06:15Z)
• Structural Disentanglement in Bilinear MLPs via Architectural Inductive Bias [0.0]
  We argue that failures arise from how models structure their internal representations during training.<n>We show analytically that bilinear parameterizations possess a non-mixing' property under gradient flow conditions.<n>Unlike pointwise nonlinear networks, multiplicative architectures are able to recover true operators aligned with the underlying algebraic structure.
  arXiv  Detail & Related papers  (2026-02-05T13:14:01Z)
• Memorization, Emergence, and Explaining Reversal Failures: A Controlled Study of Relational Semantics in LLMs [43.414287127130684]
  We propose a synthetic framework that generates text from symmetric/inverse triples, trains GPT-style autoregressive models from scratch, and evaluate memorization, logical inference, and in-context generalization.<n>We find that relational semantics emerge with sufficient logic-bearing supervision, even in shallow (2-3 layer) models, and that successful generalization aligns with stable intermediate-layer signals.
  arXiv  Detail & Related papers  (2026-01-06T11:20:38Z)
• An Equivariance Toolbox for Learning Dynamics [13.651450618432094]
  We develop a general equivariance toolbox that yields coupled first- and second-order constraints on learning dynamics.<n>At the first order, our framework unifies conservation laws and implicit-bias relations as special cases of a single identity.<n>At the second order, it provides structural predictions about curvature.
  arXiv  Detail & Related papers  (2025-12-24T23:42:07Z)
• RelTopo: Multi-Level Relational Modeling for Driving Scene Topology Reasoning [74.58385332488227]
  Road topology reasoning is critical for autonomous driving, enabling effective navigation and adherence to traffic regulations.<n>Existing methods typically focus on either lane detection or Lane-to-Lane (L2L) topology reasoning, often textitneglecting Lane-to-Traffic-element (L2T) relationships to optimize these tasks jointly.<n>We argue that relational modeling is beneficial for both perception and reasoning, as humans naturally leverage contextual relationships for road element recognition and their connectivity inference.
  arXiv  Detail & Related papers  (2025-06-16T14:40:28Z)
• How do Transformers Learn Implicit Reasoning? [67.02072851088637]
  We study how implicit multi-hop reasoning emerges by training transformers from scratch in a controlled symbolic environment.<n>We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures.
  arXiv  Detail & Related papers  (2025-05-29T17:02:49Z)
• RSCF: Relation-Semantics Consistent Filter for Entity Embedding of Knowledge Graph [5.855718296228381]
  We introduce a plug-in KGE method, Relation-Semantics Consistent Filter (RSCF)<n>Its entity transformation has three features for enhancing semantic consistency.<n>RSCF significantly outperforms state-of-the-art KGE methods.
  arXiv  Detail & Related papers  (2025-05-27T07:22:00Z)
• A Signed Graph Approach to Understanding and Mitigating Oversmoothing in GNNs [54.62268052283014]
  We present a unified theoretical perspective based on the framework of signed graphs.<n>We show that many existing strategies implicitly introduce negative edges that alter message-passing to resist oversmoothing.<n>We propose Structural Balanced Propagation (SBP), a plug-and-play method that assigns signed edges based on either labels or feature similarity.
  arXiv  Detail & Related papers  (2025-02-17T03:25:36Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networks [3.566568169425391]
  We show that with increased depth, node representations become dominated by a low-dimensional subspace that depends on the aggregation function but not on the feature transformations. For all aggregation functions, the rank of the node representations collapses, resulting in over-smoothing for particular aggregation functions.
  arXiv  Detail & Related papers  (2023-08-31T15:22:31Z)
• Bayesian network structure learning with causal effects in the presence of latent variables [6.85316573653194]
  This paper describes a hybrid structure learning algorithm, called CCHM, which combines the constraint-based part of cFCI with score-based learning. Experiments based on both randomised and well-known networks show that CCHM improves the state-of-the-art in terms of reconstructing the true ancestral graph.
  arXiv  Detail & Related papers  (2020-05-29T04:42:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.