TRACE: Trajectory Recovery for Continuous Mechanism Evolution in Causal Representation Learning

• URL: http://arxiv.org/abs/2601.21135v1
• Date: Thu, 29 Jan 2026 00:37:02 GMT
• Title: TRACE: Trajectory Recovery for Continuous Mechanism Evolution in Causal Representation Learning
• Authors: Shicheng Fan, Kun Zhang, Lu Cheng,
• Abstract summary: Temporal causal representation learning methods assume that causal mechanisms switch instantaneously between discrete domains.<n>We formalize this setting by modeling transitional mechanisms as convex combinations of finitely many atomic mechanisms.<n>We propose TRACE, a Mixture-of-Experts framework where each expert learns one atomic mechanism during training, enabling recovery of mechanism trajectories at test time.
• Score: 12.28349934159359
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal causal representation learning methods assume that causal mechanisms switch instantaneously between discrete domains, yet real-world systems often exhibit continuous mechanism transitions. For example, a vehicle's dynamics evolve gradually through a turning maneuver, and human gait shifts smoothly from walking to running. We formalize this setting by modeling transitional mechanisms as convex combinations of finitely many atomic mechanisms, governed by time-varying mixing coefficients. Our theoretical contributions establish that both the latent causal variables and the continuous mixing trajectory are jointly identifiable. We further propose TRACE, a Mixture-of-Experts framework where each expert learns one atomic mechanism during training, enabling recovery of mechanism trajectories at test time. This formulation generalizes to intermediate mechanism states never observed during training. Experiments on synthetic and real-world data demonstrate that TRACE recovers mixing trajectories with up to 0.99 correlation, substantially outperforming discrete-switching baselines.

Related papers

• Dynamical Regimes of Multimodal Diffusion Models [0.0]
  We present a theoretical framework for coupled diffusion models, using coupled Ornstein-Uhlenbeck processes as a tractable model.<n>A key prediction is the synchronization gap'', a temporal window during the reverse generative process where distinct eigenmodes stabilize at different rates.<n>We show that the coupling strength acts as a spectral filter that enforces a tunable temporal hierarchy on generation.
  arXiv  Detail & Related papers  (2026-02-04T17:16:12Z)
• Transformer Learning of Chaotic Collective Dynamics in Many-Body Systems [0.0]
  We show that a self-attention-based transformer framework provides an effective approach for modeling chaotic collective dynamics.<n>We study the one-dimensional semiclassical Holstein model, where interaction quenches induce strongly nonlinear and chaotic dynamics.<n>Our results establish self-attention as a powerful mechanism for learning effective reduced dynamics in chaotic many-body systems.
  arXiv  Detail & Related papers  (2026-01-27T01:33:33Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Kuramoto Orientation Diffusion Models [67.0711709825854]
  Orientation-rich images, such as fingerprints and textures, often exhibit coherent angular patterns.<n>Motivated by the role of phase synchronization in biological systems, we propose a score-based generative model.<n>We implement competitive results on general image benchmarks and significantly improves generation quality on orientation-dense datasets like fingerprints and textures.
  arXiv  Detail & Related papers  (2025-09-18T18:18:49Z)
• A Stage-Aware Mixture of Experts Framework for Neurodegenerative Disease Progression Modelling [6.577465893404979]
  Long-term progression of neurodegenerative diseases is commonly conceptualized as a graph diffusion process.<n>We propose a novel stage-aware Mixture Experts framework that explicitly models how different contributing mechanisms dominate at different disease stages.<n>The resulting IGND-MoE model dynamically integrates these components across temporal states, offering a principled way to understand how stage-specific pathological mechanisms contribute to progression.
  arXiv  Detail & Related papers  (2025-08-09T16:09:00Z)
• From Biased to Unbiased Dynamics: An Infinitesimal Generator Approach [24.560340485988128]
  We investigate learning the eigenfunctions of evolution operators for time-reversal invariant processes.<n>Many physical or chemical processes described by the Langevin equation involve transitions between metastable states separated by high potential barriers.<n>We propose a framework for learning from biased simulations rooted in the infinitesimal generator of the process and the associated resolvent operator.
  arXiv  Detail & Related papers  (2024-06-13T12:02:51Z)
• Compete and Compose: Learning Independent Mechanisms for Modular World Models [57.94106862271727]
  We present COMET, a modular world model which leverages reusable, independent mechanisms across different environments. COMET is trained on multiple environments with varying dynamics via a two-step process: competition and composition. We show that COMET is able to adapt to new environments with varying numbers of objects with improved sample efficiency compared to more conventional finetuning approaches.
  arXiv  Detail & Related papers  (2024-04-23T15:03:37Z)
• Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals [82.68757839524677]
  Interpretability research aims to bridge the gap between empirical success and our scientific understanding of large language models (LLMs) We propose a formulation of competition of mechanisms, which focuses on the interplay of multiple mechanisms instead of individual mechanisms. Our findings show traces of the mechanisms and their competition across various model components and reveal attention positions that effectively control the strength of certain mechanisms.
  arXiv  Detail & Related papers  (2024-02-18T17:26:51Z)
• Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction [58.67761673662716]
  Humans are highly adaptable, swiftly switching between different modes to handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. This work proposes to model two concurrent mechanisms that jointly control human motion.
  arXiv  Detail & Related papers  (2023-07-24T12:21:33Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Momentum Accelerates Evolutionary Dynamics [4.061135251278187]
  We show that momentum accelerates the convergence of evolutionary dynamics including the replicator equation and Euclidean gradient descent on populations. We also show that momentum can alter the convergence properties of these dynamics, for example by breaking the cycling associated to the rock-paper-scissors landscape.
  arXiv  Detail & Related papers  (2020-07-05T21:09:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.