Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learning

• URL: http://arxiv.org/abs/2505.03172v1
• Date: Tue, 06 May 2025 04:32:47 GMT
• Title: Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learning
• Abstract summary: Hindsight relabeling is a powerful tool for overcoming sparsity in goal-conditioned reinforcement learning.<n>We introduce Interactions (HInt), which combines interactions with hindsight relabeling to improve the sample efficiency of downstream RL.
• Score: 22.778380035995127
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hindsight relabeling is a powerful tool for overcoming sparsity in goal-conditioned reinforcement learning (GCRL), especially in certain domains such as navigation and locomotion. However, hindsight relabeling can struggle in object-centric domains. For example, suppose that the goal space consists of a robotic arm pushing a particular target block to a goal location. In this case, hindsight relabeling will give high rewards to any trajectory that does not interact with the block. However, these behaviors are only useful when the object is already at the goal -- an extremely rare case in practice. A dataset dominated by these kinds of trajectories can complicate learning and lead to failures. In object-centric domains, one key intuition is that meaningful trajectories are often characterized by object-object interactions such as pushing the block with the gripper. To leverage this intuition, we introduce Hindsight Relabeling using Interactions (HInt), which combines interactions with hindsight relabeling to improve the sample efficiency of downstream RL. However because interactions do not have a consensus statistical definition tractable for downstream GCRL, we propose a definition of interactions based on the concept of null counterfactual: a cause object is interacting with a target object if, in a world where the cause object did not exist, the target object would have different transition dynamics. We leverage this definition to infer interactions in Null Counterfactual Interaction Inference (NCII), which uses a "nulling'' operation with a learned model to infer interactions. NCII is able to achieve significantly improved interaction inference accuracy in both simple linear dynamics domains and dynamic robotic domains in Robosuite, Robot Air Hockey, and Franka Kitchen and HInt improves sample efficiency by up to 4x.

Related papers

• FineHOI: Part-Aware Dense Representations for Zero-Shot Human-Object Interaction Detection [26.006274699996027]
  FineHOI is a zero-shot HOI framework that explicitly models interactions from dense patch-level features.<n>Our approach is motivated by the observation that human-object interactions are defined by localized spatial relationships.<n>Experiments demonstrate that FineHOI consistently outperforms existing zero-shot HOI methods.
  arXiv  Detail & Related papers  (2026-09-05T07:53:11Z)
• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• Riding the Shifting Potential: When Reactive Control Suffices for Multi-Goal Behavior [7.648784748888186]
  We exploit encodings that fail to reflect how objectives currently are.<n>We demonstrate this in two domains where conflicts interact by extending the nullspace.<n>The same formulation transfers directly to a real robot with additional control.
  arXiv  Detail & Related papers  (2026-05-26T17:24:46Z)
• Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors [51.096845970243855]
  Whole-body Humanoid-Object Interaction (HOI) is bottlenecked by the scarcity of high-fidelity 3D data.<n>We propose Imagine2Real, a zero-shot HOI framework for flexible, geometry-free interaction.
  arXiv  Detail & Related papers  (2026-05-21T10:15:39Z)
• RegFormer: Transferable Relational Grounding for Efficient Weakly-Supervised Human-Object Interaction Detection [38.362111975504696]
  Weakly-supervised Human-Object Interaction (HOI) detection is essential for scene understanding.<n>RegFormer is a versatile interaction recognition module for instance-level HOI reasoning.<n>Our experiments and analyses demonstrate that RegFormer effectively learns spatial cues for instance-level interaction reasoning.
  arXiv  Detail & Related papers  (2026-04-01T05:47:35Z)
• InterReal: A Unified Physics-Based Imitation Framework for Learning Human-Object Interaction Skills [16.31202379413011]
  We develop a unified physics-based imitation learning framework for Real-world human-object Interaction (HOI) control.<n>InterReal enables humanoid robots to track HOI reference motions, facilitating the learning of fine-grained interactive skills.<n>A meta-policy guided by critical tracking error metrics explores and allocates reward signals to the low-level reinforcement learning objective.
  arXiv  Detail & Related papers  (2026-03-08T07:44:35Z)
• Funnel-HOI: Top-Down Perception for Zero-Shot HOI Detection [3.656114607436271]
  Human-object interaction detection (HOID) refers to localizing interactive human-object pairs in images and identifying the interactions.<n>We build a top-down framework named Funnel-HOI inspired by the human tendency to grasp well-defined concepts first and then associate them with abstract concepts during scene understanding.<n>A novel asymmetric co-attention mechanism mines these cues utilizing multimodal information (incorporating zero-shot capabilities) and yields stronger interaction representations at the encoder level.
  arXiv  Detail & Related papers  (2025-07-16T20:47:24Z)
• Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control [72.00655365269]
  We present RoboMaster, a novel framework that models inter-object dynamics through a collaborative trajectory formulation.<n>Unlike prior methods that decompose objects, our core is to decompose the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction.<n>Our method outperforms existing approaches, establishing new state-of-the-art performance in trajectory-controlled video generation for robotic manipulation.
  arXiv  Detail & Related papers  (2025-06-02T17:57:06Z)
• Object-Centric Latent Action Learning [70.3173534658611]
  We propose a novel object-centric latent action learning framework that centers on objects rather than pixels.<n>We leverage self-supervised object-centric pretraining to disentangle action-related and distracting dynamics.<n>Our results show that object-centric pretraining mitigates the negative effects of distractors by 50%.
  arXiv  Detail & Related papers  (2025-02-13T11:27:05Z)
• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [57.942404069484134]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered.<n>Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.<n>We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)
• Cross-Embodied Affordance Transfer through Learning Affordance Equivalences [6.828097734917722]
  We propose a deep neural network model that unifies objects, actions, and effects into a single latent vector in a common latent space that we call the affordance space. Our model does not learn the behavior of individual objects acted upon by a single agent. Affordance Equivalence facilitates not only action generalization over objects but also Cross Embodiment transfer linking actions of different robots.
  arXiv  Detail & Related papers  (2024-04-24T05:07:36Z)
• Neural-Logic Human-Object Interaction Detection [67.4993347702353]
  We present L OGIC HOI, a new HOI detector that leverages neural-logic reasoning and Transformer to infer feasible interactions between entities. Specifically, we modify the self-attention mechanism in vanilla Transformer, enabling it to reason over the human, action, object> triplet and constitute novel interactions. We formulate these two properties in first-order logic and ground them into continuous space to constrain the learning process of our approach, leading to improved performance and zero-shot generalization capabilities.
  arXiv  Detail & Related papers  (2023-11-16T11:47:53Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)
• NIFT: Neural Interaction Field and Template for Object Manipulation [24.42098331719611]
  NIFT is a descriptive and robust interaction representation to facilitate imitation learning. NIF is a neural field which encodes the relationship between each spatial point and a given object. NIT effectively guides the feature matching in the NIFs of the new object instances to optimize the object poses.
  arXiv  Detail & Related papers  (2022-10-20T03:35:05Z)
• Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detection [22.3445174577181]
  Human-Object Interaction (HOI) in images is an important step towards high-level visual comprehension. We propose a novel plug-and-play Object-wise Debiasing Memory (ODM) method for re-balancing the distribution of interactions under detected objects. Our method brings consistent and significant improvements over baselines, especially on rare interactions under each object.
  arXiv  Detail & Related papers  (2022-07-06T01:55:28Z)
• Domain Knowledge Driven Pseudo Labels for Interpretable Goal-Conditioned Interactive Trajectory Prediction [29.701029725302586]
  We study the joint trajectory prediction problem with the goal-conditioned framework. We introduce a conditional-variational-autoencoder-based (CVAE) model to explicitly encode different interaction modes into the latent space. We propose a novel approach to avoid KL vanishing and induce an interpretable interactive latent space with pseudo labels.
  arXiv  Detail & Related papers  (2022-03-28T21:41:21Z)
• INVIGORATE: Interactive Visual Grounding and Grasping in Clutter [56.00554240240515]
  INVIGORATE is a robot system that interacts with human through natural language and grasps a specified object in clutter. We train separate neural networks for object detection, for visual grounding, for question generation, and for OBR detection and grasping. We build a partially observable Markov decision process (POMDP) that integrates the learned neural network modules.
  arXiv  Detail & Related papers  (2021-08-25T07:35:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.