Better Decisions through the Right Causal World Model

• URL: http://arxiv.org/abs/2504.07257v1
• Date: Wed, 09 Apr 2025 20:29:13 GMT
• Title: Better Decisions through the Right Causal World Model
• Authors: Elisabeth Dillies, Quentin Delfosse, Jannis Blüml, Raban Emunds, Florian Peter Busch, Kristian Kersting,
• Abstract summary: Causal Object-centric Model Extraction Tool (COMET) is a novel algorithm designed to learn the exact interpretable causal world models (CWMs)<n>Our results, validated in Atari environments such as Pong and Freeway, demonstrate the accuracy and robustness of COMET.
• Score: 17.623937562865617
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) agents have shown remarkable performances in various environments, where they can discover effective policies directly from sensory inputs. However, these agents often exploit spurious correlations in the training data, resulting in brittle behaviours that fail to generalize to new or slightly modified environments. To address this, we introduce the Causal Object-centric Model Extraction Tool (COMET), a novel algorithm designed to learn the exact interpretable causal world models (CWMs). COMET first extracts object-centric state descriptions from observations and identifies the environment's internal states related to the depicted objects' properties. Using symbolic regression, it models object-centric transitions and derives causal relationships governing object dynamics. COMET further incorporates large language models (LLMs) for semantic inference, annotating causal variables to enhance interpretability. By leveraging these capabilities, COMET constructs CWMs that align with the true causal structure of the environment, enabling agents to focus on task-relevant features. The extracted CWMs mitigate the danger of shortcuts, permitting the development of RL systems capable of better planning and decision-making across dynamic scenarios. Our results, validated in Atari environments such as Pong and Freeway, demonstrate the accuracy and robustness of COMET, highlighting its potential to bridge the gap between object-centric reasoning and causal inference in reinforcement learning.

Related papers

• Towards Causal Model-Based Policy Optimization [0.24578723416255752]
  We introduce Causal Model-Based Policy Optimization (C-MBPO)<n>C-MBPO is a novel framework that integrates causal learning into the Model-Based Reinforcement Learning pipeline.<n>We show that C-MBPO can be shown to be robust to a class of distributional shifts that affect spurious, non-causal relationships in the dynamics.
  arXiv  Detail & Related papers  (2025-03-12T18:09:02Z)
• Towards Empowerment Gain through Causal Structure Learning in Model-Based RL [35.933469787075]
  We propose a novel framework, Empowerment through Causal Learning (ECL), to improve learning efficiency and controllability.<n>ECL operates by first training a causal dynamics model of the environment based on collected data.<n>We then maximize empowerment under the causal structure for exploration, simultaneously using data gathered through exploration to update causal dynamics model to be more controllable.
  arXiv  Detail & Related papers  (2025-02-14T10:59:09Z)
• SPARTAN: A Sparse Transformer Learning Local Causation [63.29645501232935]
  Causal structures play a central role in world models that flexibly adapt to changes in the environment. We present the SPARse TrANsformer World model (SPARTAN), a Transformer-based world model that learns local causal structures between entities in a scene. By applying sparsity regularisation on the attention pattern between object-factored tokens, SPARTAN identifies sparse local causal models that accurately predict future object states.
  arXiv  Detail & Related papers  (2024-11-11T11:42:48Z)
• Language Agents Meet Causality -- Bridging LLMs and Causal World Models [50.79984529172807]
  We propose a framework that integrates causal representation learning with large language models. This framework learns a causal world model, with causal variables linked to natural language expressions. We evaluate the framework on causal inference and planning tasks across temporal scales and environmental complexities.
  arXiv  Detail & Related papers  (2024-10-25T18:36:37Z)
• Variable-Agnostic Causal Exploration for Reinforcement Learning [56.52768265734155]
  We introduce a novel framework, Variable-Agnostic Causal Exploration for Reinforcement Learning (VACERL) Our approach automatically identifies crucial observation-action steps associated with key variables using attention mechanisms. It constructs the causal graph connecting these steps, which guides the agent towards observation-action pairs with greater causal influence on task completion.
  arXiv  Detail & Related papers  (2024-07-17T09:45:27Z)
• Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy [38.86867078596718]
  We consider explicitly modeling the generation process of states with the graphical causal model. We formulate the causal structure updating into the RL interaction process with active intervention learning of the environment.
  arXiv  Detail & Related papers  (2024-02-07T14:09:34Z)
• Causal Flow-based Variational Auto-Encoder for Disentangled Causal Representation Learning [1.4875602190483512]
  Disentangled representation learning aims to learn representations where each dimension corresponds to an underlying generative factor.<n>We propose the Disentangled Causal Variational Auto-Encoder (DCVAE), a novel supervised VAE framework that integrates causal flows into the representation learning process.<n>We evaluate DCVAE on both synthetic and real-world datasets, demonstrating its superior ability in causal disentanglement and intervention experiments.
  arXiv  Detail & Related papers  (2023-04-18T14:26:02Z)
• Denoised MDPs: Learning World Models Better Than the World Itself [94.74665254213588]
  This work categorizes information out in the wild into four types based on controllability and relation with reward, and formulates useful information as that which is both controllable and reward-relevant. Experiments on variants of DeepMind Control Suite and RoboDesk demonstrate superior performance of our denoised world model over using raw observations alone.
  arXiv  Detail & Related papers  (2022-06-30T17:59:49Z)
• INFOrmation Prioritization through EmPOWERment in Visual Model-Based RL [90.06845886194235]
  We propose a modified objective for model-based reinforcement learning (RL) We integrate a term inspired by variational empowerment into a state-space model based on mutual information. We evaluate the approach on a suite of vision-based robot control tasks with natural video backgrounds.
  arXiv  Detail & Related papers  (2022-04-18T23:09:23Z)
• Invariant Causal Prediction for Block MDPs [106.63346115341862]
  Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. We propose a method of invariant prediction to learn model-irrelevance state abstractions (MISA) that generalize to novel observations in the multi-environment setting.
  arXiv  Detail & Related papers  (2020-03-12T21:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.