Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks

• URL: http://arxiv.org/abs/2511.08086v2
• Date: Fri, 14 Nov 2025 17:15:33 GMT
• Title: Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks
• Authors: Muthukumar Pandaram, Jakob Hollenstein, David Drexel, Samuele Tosatto, Antonio Rodríguez-Sánchez, Justus Piater,
• Abstract summary: We study whether causal graphs of environment dynamics are sparse, (ii) whether such sparsity is state-dependent, and (iii) whether local system dynamics change sparsely.<n>Our results indicate that global sparsity is rare, but instead the tasks show local, state-dependent sparsity in their dynamics.
• Score: 2.093690479400306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of learned dynamics models, also known as world models, can improve the sample efficiency of reinforcement learning. Recent work suggests that the underlying causal graphs of such dynamics models are sparsely connected, with each of the future state variables depending only on a small subset of the current state variables, and that learning may therefore benefit from sparsity priors. Similarly, temporal sparsity, i.e. sparsely and abruptly changing local dynamics, has also been proposed as a useful inductive bias. In this work, we critically examine these assumptions by analyzing ground-truth dynamics from a set of robotic reinforcement learning environments in the MuJoCo Playground benchmark suite, aiming to determine whether the proposed notions of state and temporal sparsity actually tend to hold in typical reinforcement learning tasks. We study (i) whether the causal graphs of environment dynamics are sparse, (ii) whether such sparsity is state-dependent, and (iii) whether local system dynamics change sparsely. Our results indicate that global sparsity is rare, but instead the tasks show local, state-dependent sparsity in their dynamics and this sparsity exhibits distinct structures, appearing in temporally localized clusters (e.g., during contact events) and affecting specific subsets of state dimensions. These findings challenge common sparsity prior assumptions in dynamics learning, emphasizing the need for grounded inductive biases that reflect the state-dependent sparsity structure of real-world dynamics.

Related papers

• Causal Structure Learning for Dynamical Systems with Theoretical Score Analysis [7.847876045564289]
  Real world systems evolve in continuous-time according to their underlying causal relationships, yet their dynamics are often unknown.<n>We propose CaDyT, a novel method for causal discovery on dynamical systems.<n>Our experiments show that CaDyT outperforms state-of-the-art methods on both regularly and irregularly-sampled data.
  arXiv  Detail & Related papers  (2025-12-16T12:41:22Z)
• Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models [71.63194926457119]
  We introduce Dynamical Diffusion (DyDiff), a theoretically sound framework that incorporates temporally aware forward and reverse processes.<n>Experiments across scientifictemporal forecasting, video prediction, and time series forecasting demonstrate that Dynamical Diffusion consistently improves performance in temporal predictive tasks.
  arXiv  Detail & Related papers  (2025-03-02T16:10:32Z)
• SPARTAN: A Sparse Transformer Learning Local Causation [63.29645501232935]
  Causal structures play a central role in world models that flexibly adapt to changes in the environment. We present the SPARse TrANsformer World model (SPARTAN), a Transformer-based world model that learns local causal structures between entities in a scene. By applying sparsity regularisation on the attention pattern between object-factored tokens, SPARTAN identifies sparse local causal models that accurately predict future object states.
  arXiv  Detail & Related papers  (2024-11-11T11:42:48Z)
• Reinforcement Learning under Latent Dynamics: Toward Statistical and Algorithmic Modularity [51.40558987254471]
  Real-world applications of reinforcement learning often involve environments where agents operate on complex, high-dimensional observations. This paper addresses the question of reinforcement learning under $textitgeneral$ latent dynamics from a statistical and algorithmic perspective.
  arXiv  Detail & Related papers  (2024-10-23T14:22:49Z)
• Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training [7.5041863920639456]
  Machine learning systems often acquire biases by leveraging undesired features in the data, impacting accuracy across different sub-populations.<n>This paper explores the evolution of bias in a teacher-student setup modeling different data sub-populations with a Gaussian-mixture model.<n>Applying our findings to fairness and robustness, we delineate how and when heterogeneous data and spurious features can generate and amplify bias.
  arXiv  Detail & Related papers  (2024-05-28T15:50:10Z)
• Loss Dynamics of Temporal Difference Reinforcement Learning [36.772501199987076]
  We study the case learning curves for temporal difference learning of a value function with linear function approximators. We study how learning dynamics and plateaus depend on feature structure, learning rate, discount factor, and reward function.
  arXiv  Detail & Related papers  (2023-07-10T18:17:50Z)
• Sparsity in Continuous-Depth Neural Networks [2.969794498016257]
  We study the influence of weight and feature sparsity on forecasting and on identifying the underlying dynamical laws. We curate real-world datasets consisting of human motion capture and human hematopoiesis single-cell RNA-seq data.
  arXiv  Detail & Related papers  (2022-10-26T12:48:12Z)
• Causal Dynamics Learning for Task-Independent State Abstraction [61.707048209272884]
  We introduce Causal Dynamics Learning for Task-Independent State Abstraction (CDL) CDL learns a theoretically proved causal dynamics model that removes unnecessary dependencies between state variables and the action. A state abstraction can then be derived from the learned dynamics.
  arXiv  Detail & Related papers  (2022-06-27T17:02:53Z)
• Deep Active Learning by Leveraging Training Dynamics [57.95155565319465]
  We propose a theory-driven deep active learning method (dynamicAL) which selects samples to maximize training dynamics. We show that dynamicAL not only outperforms other baselines consistently but also scales well on large deep learning models.
  arXiv  Detail & Related papers  (2021-10-16T16:51:05Z)
• LEADS: Learning Dynamical Systems that Generalize Across Environments [12.024388048406587]
  We propose LEADS, a novel framework that leverages the commonalities and discrepancies among known environments to improve model generalization. We show that this new setting can exploit knowledge extracted from environment-dependent data and improves generalization for both known and novel environments.
  arXiv  Detail & Related papers  (2021-06-08T17:28:19Z)
• Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning [124.9856253431878]
  We decompose the task of learning a global dynamics model into two stages: (a) learning a context latent vector that captures the local dynamics, then (b) predicting the next state conditioned on it. In order to encode dynamics-specific information into the context latent vector, we introduce a novel loss function that encourages the context latent vector to be useful for predicting both forward and backward dynamics. The proposed method achieves superior generalization ability across various simulated robotics and control tasks, compared to existing RL schemes.
  arXiv  Detail & Related papers  (2020-05-14T08:10:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.