Towards intervention-centric causal reasoning in learning agents

• URL: http://arxiv.org/abs/2005.12968v1
• Date: Tue, 26 May 2020 18:53:04 GMT
• Title: Towards intervention-centric causal reasoning in learning agents
• Abstract summary: We show how a meta-learning approach can be used to perform causal learning in high-dimensional environments. A meta-reinforcement learning algorithm is used to learn relationships that transfer on observational causal learning tasks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interventions are central to causal learning and reasoning. Yet ultimately an intervention is an abstraction: an agent embedded in a physical environment (perhaps modeled as a Markov decision process) does not typically come equipped with the notion of an intervention -- its action space is typically ego-centric, without actions of the form `intervene on X'. Such a correspondence between ego-centric actions and interventions would be challenging to hard-code. It would instead be better if an agent learnt which sequence of actions allow it to make targeted manipulations of the environment, and learnt corresponding representations that permitted learning from observation. Here we show how a meta-learning approach can be used to perform causal learning in this challenging setting, where the action-space is not a set of interventions and the observation space is a high-dimensional space with a latent causal structure. A meta-reinforcement learning algorithm is used to learn relationships that transfer on observational causal learning tasks. This work shows how advances in deep reinforcement learning and meta-learning can provide intervention-centric causal learning in high-dimensional environments with a latent causal structure.

Related papers

• An Introduction to Causal Reinforcement Learning [58.680653905480284]
  Causal inference and reinforcement learning operate over different aspects of the same building block, counterfactual relations.<n>We put under a unifying treatment different modes of learning, including online, off-policy, and causal calculus learning.<n>Specifically, we introduce and discuss through causal lenses generalized policy learning, where to intervene, imitation learning, and counterfactual learning.
  arXiv  Detail & Related papers  (2026-06-23T05:28:33Z)
• From Perception to Action: An Interactive Benchmark for Vision Reasoning [51.11355591375073]
  Causal Hierarchy of Actions and Interactions (CHAIN) benchmark designed to evaluate whether models can understand, plan, and execute structured action sequences grounded in physical constraints.<n> CHAIN shifts evaluation from passive perception to active problem solving, spanning tasks such as interlocking mechanical puzzles and 3D stacking and packing.<n>Our results show that top-performing models still struggle to internalize physical structure and causal constraints, often failing to produce reliable long-horizon plans and cannot robustly translate perceived structure into effective actions.
  arXiv  Detail & Related papers  (2026-02-24T15:33:02Z)
• Act, Sense, Act: Learning Non-Markovian Active Perception Strategies from Large-Scale Egocentric Human Data [30.493131697188627]
  We formalize active perception as a non-Markovian process driven by information gain and decision branching.<n>We introduce CoMe-VLA, a cognitive and memory-aware vision-language-action framework.<n>Our framework integrates a cognitive auxiliary head for autonomous sub-task transitions and a dual-track memory system to maintain consistent self and environmental awareness.
  arXiv  Detail & Related papers  (2026-02-04T14:28:14Z)
• Towards Error Centric Intelligence I, Beyond Observational Learning [0.0]
  We argue that progress toward AGI is theory limited rather than data or scale limited.<n>We begin by laying foundations, definitions of knowledge, learning, intelligence, counterfactual competence and AGI.<n>We recast the problem as three questions about how explicit and implicit errors evolve under an agent's actions.
  arXiv  Detail & Related papers  (2025-10-16T20:33:55Z)
• Behavioral Exploration: Learning to Explore via In-Context Adaptation [53.92981562916783]
  We train a long-context generative model to predict expert actions conditioned on a context of past observations and a measure of how exploratory'' the expert's behaviors are relative to this context.<n>This enables the model to not only mimic the behavior of an expert, but also, by feeding its past history of interactions into its context, to select different expert behaviors than what have been previously selected.<n>We demonstrate the effectiveness of our method in both simulated locomotion and manipulation settings, as well as on real-world robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-07-11T21:36:19Z)
• MILE: Model-based Intervention Learning [0.0]
  We show that it is possible to learn a policy with just a handful of expert interventions. Our key insight is that it is possible to get crucial information about the quality of the current state and the optimality of the chosen action from expert feedback.
  arXiv  Detail & Related papers  (2025-02-19T08:15:16Z)
• Dynamic planning in hierarchical active inference [0.0]
  We refer to the ability of the human brain to infer and impose motor trajectories related to cognitive decisions. This study focuses on the topic of dynamic planning in active inference.
  arXiv  Detail & Related papers  (2024-02-18T17:32:53Z)
• Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations [62.48505112245388]
  We take an in-depth look at the causal awareness of modern representations of agent interactions. We show that recent representations are already partially resilient to perturbations of non-causal agents. We propose a metric learning approach that regularizes latent representations with causal annotations.
  arXiv  Detail & Related papers  (2023-12-07T18:57:03Z)
• When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning [57.53138994155612]
  A long-term goal of reinforcement learning is to design agents that can autonomously interact and learn in the world. A critical challenge is the presence of irreversible states which require external assistance to recover from, such as when a robot arm has pushed an object off of a table. We propose an algorithm that efficiently learns to detect and avoid states that are irreversible, and proactively asks for help in case the agent does enter them.
  arXiv  Detail & Related papers  (2022-10-19T17:57:24Z)
• Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions [51.71245032890532]
  We propose methods enabling an agent acting upon the world to learn internal representations of sensory information consistent with actions that modify it. In contrast to existing work, our approach does not require prior knowledge of the group and does not restrict the set of actions the agent can perform.
  arXiv  Detail & Related papers  (2022-07-25T11:22:48Z)
• Towards a Grounded Theory of Causation for Embodied AI [12.259552039796027]
  Existing frameworks give no indication as to which behaviour policies or physical transformations of state space shall count as interventions. The framework sketched in this paper describes actions as transformations of state space, for instance induced by an agent running a policy. This makes it possible to describe in a uniform way both transformations of the micro-state space and abstract models thereof.
  arXiv  Detail & Related papers  (2022-06-28T12:56:43Z)
• Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams [64.82800502603138]
  This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream. The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations. Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream.
  arXiv  Detail & Related papers  (2022-04-26T09:52:31Z)
• Inference of Affordances and Active Motor Control in Simulated Agents [0.5161531917413706]
  We introduce an output-probabilistic, temporally predictive, modular artificial neural network architecture. We show that our architecture develops latent states that can be interpreted as affordance maps. In combination with active inference, we show that flexible, goal-directed behavior can be invoked.
  arXiv  Detail & Related papers  (2022-02-23T14:13:04Z)
• Learning to reason about and to act on physical cascading events [72.90358875111883]
  We introduce a new supervised learning setup called em Cascade where an agent is shown a video of a physically simulated dynamic scene. The agent is asked to intervene and trigger a cascade of events, such that the system reaches a "counterfactual" goal. We demonstrate that our approach learns to effectively follow instructions to intervene in previously unseen complex scenes.
  arXiv  Detail & Related papers  (2022-02-02T16:17:42Z)
• Shaking the foundations: delusions in sequence models for interaction and control [45.34593341136043]
  We show that sequence models "lack the understanding of the cause and effect of their actions" leading them to draw incorrect inferences due to auto-suggestive delusions. We show that in supervised learning, one can teach a system to condition or intervene on data by training with factual and counterfactual error signals respectively.
  arXiv  Detail & Related papers  (2021-10-20T23:31:05Z)
• Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities [54.615003524001686]
  This paper proposes a novel approach for unsupervised sub-action learning in complex activities. The proposed method maps both visual and temporal representations to a latent space where the sub-actions are learnt discriminatively. We show that the proposed combination of visual-temporal embedding and discriminative latent concepts allow to learn robust action representations in an unsupervised setting.
  arXiv  Detail & Related papers  (2021-04-30T20:07:27Z)
• On the Sensory Commutativity of Action Sequences for Embodied Agents [2.320417845168326]
  We study perception for embodied agents under the mathematical formalism of group theory. We introduce the Sensory Commutativity Probability criterion which measures how much an agent's degree of freedom affects the environment. We empirically illustrate how SCP and the commutative properties of action sequences can be used to learn about objects in the environment.
  arXiv  Detail & Related papers  (2020-02-13T16:58:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.