Dynamics Reveals Structure: Challenging the Linear Propagation Assumption

• URL: http://arxiv.org/abs/2601.21601v1
• Date: Thu, 29 Jan 2026 12:08:00 GMT
• Title: Dynamics Reveals Structure: Challenging the Linear Propagation Assumption
• Abstract summary: We study whether local updates coherently propagate to logical consequences.<n>For negation and converse, we prove that guaranteeing direction-agnostic first-order propagation requires a tensor factorization separating entity-pair context from relation content.<n>We show that composition reduces to conjunction, and prove that any conjunction well-defined on linear features must be bilinear.
• Score: 23.984006583336498
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks adapt through first-order parameter updates, yet it remains unclear whether such updates preserve logical coherence. We investigate the geometric limits of the Linear Propagation Assumption (LPA), the premise that local updates coherently propagate to logical consequences. To formalize this, we adopt relation algebra and study three core operations on relations: negation flips truth values, converse swaps argument order, and composition chains relations. For negation and converse, we prove that guaranteeing direction-agnostic first-order propagation necessitates a tensor factorization separating entity-pair context from relation content. However, for composition, we identify a fundamental obstruction. We show that composition reduces to conjunction, and prove that any conjunction well-defined on linear features must be bilinear. Since bilinearity is incompatible with negation, this forces the feature map to collapse. These results suggest that failures in knowledge editing, the reversal curse, and multi-hop reasoning may stem from common structural limitations inherent to the LPA.

Related papers

• Commutator-Induced Uncertainty in VAEs [20.84327211490012]
  Variational autoencoders (VAEs) often struggle to represent non-commutative structure in learned latent spaces.<n>We introduce a Lie Group VAE framework that combines geometric and algebraic perspectives on uncertainty.<n>We evaluate the framework on dSprites, 3DShapes, 3DCars, and CelebA against generic and symmetry-aware baselines.
  arXiv  Detail & Related papers  (2026-05-22T10:03:46Z)
• Geometric Factual Recall in Transformers [57.48371649045765]
  A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring memorization counts that scale linearly with the number of facts.<n>We develop a theoretical and empirical account of an alternative, emphgeometric form of memorization in which learned embeddings encode structure directly.<n>In a controlled setting where a single-layer transformer must memorize random vectors from subjects to a shared attribute set, we prove that a logarithmic embedding dimension suffices.<n>We extend these results to the multi-hop setting, providing constructions with and without chain-of-thought that exhibit a provable
  arXiv  Detail & Related papers  (2026-05-12T17:22:22Z)
• A Mathematical Theory of Ranking [0.0]
  Ranking systems produce ordered lists from scalar scores, yet the ranking itself depends only on pairwise comparisons.<n>We develop a mathematical theory that takes this observation seriously, centering the analysis on pairwise margins rather than absolute scores.
  arXiv  Detail & Related papers  (2026-04-09T17:00:49Z)
• The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations [6.414826816896125]
  Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries.<n>We argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion"<n>We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing.
  arXiv  Detail & Related papers  (2026-03-24T13:41:57Z)
• CO-EVOLVE: Bidirectional Co-Evolution of Graph Structure and Semantics for Heterophilous Learning [2.5456314910104836]
  CO-EVOLVE is a dual-view co-evolution framework that treats graph topology and semantic embeddings as dynamic, mutually reinforcing latent variables.<n>Experiments on public benchmarks demonstrate that CO-EVOLVE significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-20T03:15:17Z)
• CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching [50.65932158912512]
  We propose a new causal reasoning benchmark, CausalFlip, to encourage the development of new large language models.<n>CaulFlip consists of causal judgment questions built over event triples that could form different confounder, chain, and collider relations.<n>We evaluate LLMs under multiple training paradigms, including answer-only training, explicit Chain-of-Thought supervision, and a proposed internalized causal reasoning approach.
  arXiv  Detail & Related papers  (2026-02-23T18:06:15Z)
• Structural Disentanglement in Bilinear MLPs via Architectural Inductive Bias [0.0]
  We argue that failures arise from how models structure their internal representations during training.<n>We show analytically that bilinear parameterizations possess a non-mixing' property under gradient flow conditions.<n>Unlike pointwise nonlinear networks, multiplicative architectures are able to recover true operators aligned with the underlying algebraic structure.
  arXiv  Detail & Related papers  (2026-02-05T13:14:01Z)
• Memorization, Emergence, and Explaining Reversal Failures: A Controlled Study of Relational Semantics in LLMs [43.414287127130684]
  We propose a synthetic framework that generates text from symmetric/inverse triples, trains GPT-style autoregressive models from scratch, and evaluate memorization, logical inference, and in-context generalization.<n>We find that relational semantics emerge with sufficient logic-bearing supervision, even in shallow (2-3 layer) models, and that successful generalization aligns with stable intermediate-layer signals.
  arXiv  Detail & Related papers  (2026-01-06T11:20:38Z)
• An Equivariance Toolbox for Learning Dynamics [13.651450618432094]
  We develop a general equivariance toolbox that yields coupled first- and second-order constraints on learning dynamics.<n>At the first order, our framework unifies conservation laws and implicit-bias relations as special cases of a single identity.<n>At the second order, it provides structural predictions about curvature.
  arXiv  Detail & Related papers  (2025-12-24T23:42:07Z)
• RelTopo: Multi-Level Relational Modeling for Driving Scene Topology Reasoning [74.58385332488227]
  Road topology reasoning is critical for autonomous driving, enabling effective navigation and adherence to traffic regulations.<n>Existing methods typically focus on either lane detection or Lane-to-Lane (L2L) topology reasoning, often textitneglecting Lane-to-Traffic-element (L2T) relationships to optimize these tasks jointly.<n>We argue that relational modeling is beneficial for both perception and reasoning, as humans naturally leverage contextual relationships for road element recognition and their connectivity inference.
  arXiv  Detail & Related papers  (2025-06-16T14:40:28Z)
• How do Transformers Learn Implicit Reasoning? [67.02072851088637]
  We study how implicit multi-hop reasoning emerges by training transformers from scratch in a controlled symbolic environment.<n>We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures.
  arXiv  Detail & Related papers  (2025-05-29T17:02:49Z)
• RSCF: Relation-Semantics Consistent Filter for Entity Embedding of Knowledge Graph [5.855718296228381]
  We introduce a plug-in KGE method, Relation-Semantics Consistent Filter (RSCF)<n>Its entity transformation has three features for enhancing semantic consistency.<n>RSCF significantly outperforms state-of-the-art KGE methods.
  arXiv  Detail & Related papers  (2025-05-27T07:22:00Z)
• A Signed Graph Approach to Understanding and Mitigating Oversmoothing in GNNs [54.62268052283014]
  We present a unified theoretical perspective based on the framework of signed graphs.<n>We show that many existing strategies implicitly introduce negative edges that alter message-passing to resist oversmoothing.<n>We propose Structural Balanced Propagation (SBP), a plug-and-play method that assigns signed edges based on either labels or feature similarity.
  arXiv  Detail & Related papers  (2025-02-17T03:25:36Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networks [3.566568169425391]
  We show that with increased depth, node representations become dominated by a low-dimensional subspace that depends on the aggregation function but not on the feature transformations. For all aggregation functions, the rank of the node representations collapses, resulting in over-smoothing for particular aggregation functions.
  arXiv  Detail & Related papers  (2023-08-31T15:22:31Z)
• Bayesian network structure learning with causal effects in the presence of latent variables [6.85316573653194]
  This paper describes a hybrid structure learning algorithm, called CCHM, which combines the constraint-based part of cFCI with score-based learning. Experiments based on both randomised and well-known networks show that CCHM improves the state-of-the-art in terms of reconstructing the true ancestral graph.
  arXiv  Detail & Related papers  (2020-05-29T04:42:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.