FOCI Policy: Focus on Object-Centric Interactions for Relational Manipulation Policies

• URL: http://arxiv.org/abs/2609.08743v1
• Date: Tue, 08 Sep 2026 13:35:56 GMT
• Title: FOCI Policy: Focus on Object-Centric Interactions for Relational Manipulation Policies
• Abstract summary: Object-centric manipulation policies improve generalization by modeling object motion instead of directly predicting robot actions.<n>We propose textscFoci Policy, an interaction-centric framework that achieves a two-fold abstraction.<n>Experiments on RLBench, COLOSSEUM, and real-world tasks show that textscFoci Policy achieves strong performance with substantially less training data than prior object-centric and action-centric policies.
• Score: 6.557127548028966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object-centric manipulation policies improve generalization by modeling object motion instead of directly predicting robot actions. However, existing methods are often limited by representations which are either too simplistic to capture interaction dynamics or too dense to learn efficiently. We observe that many rigid relational manipulation tasks are governed by short interaction phases where the relative motion between task-relevant objects is tightly constrained. Based on this observation, we propose \textsc{Foci Policy}, an interaction-centric framework that achieves a two-fold abstraction: (1) temporally, by automatically extracting compact interaction segments from demonstrations;(2) spatially, by representing skills as relative $SE(3)$ motion between task-relevant objects, yielding invariance to scene configurations and robot embodiment. Experiments on RLBench, COLOSSEUM, and real-world tasks show that \textsc{Foci Policy} achieves strong performance with substantially less training data than prior object-centric and action-centric policies. These results suggest that modeling object-object interactions provides a simple and efficient inductive bias for rigid relational manipulation. Project page: \href{https://fitz0401.github.io/foci-page/}{fitz0401.github.io/foci-page/}.

Related papers

• Joint Discovery of Object and Action Symbols through Effect Prediction for Robotic Manipulation Planning [1.4701783773624093]
  We propose a model that jointly discovers high-level manipulation primitives and object categories.<n>We leverage a discrete planning method that uses intermediate steps in the predicted effect trajectory to enable partial action executions.<n>We conduct experiments on tabletop repositioning and stacking tasks, and confirm that our effect-driven planning approach outperforms both a state-of-the-art method and a visual-based alternative.
  arXiv  Detail & Related papers  (2026-06-22T11:58:11Z)
• TriRelVLA: Triadic Relational Structure for Generalizable Embodied Manipulation [54.81449795163812]
  Vision-language--action (VLA) models perform well on training-seen robotic tasks but struggle to generalize to unseen scenes and objects.<n>We propose TriRelVLA, a triadic relational VLA framework for generalizable embodied manipulation.<n> Experiments show strong performance on fine-tuned tasks and clear gains in cross-scene, cross-object, and cross-task generalization.
  arXiv  Detail & Related papers  (2026-05-07T05:57:49Z)
• rt-RISeg: Real-Time Model-Free Robot Interactive Segmentation for Active Instance-Level Object Understanding [7.264443471771696]
  We propose a novel real-time interactive perception framework, rt-RISeg, that continuously segments unseen objects by robot interactions.<n>We demonstrate that the relative rotational and linear velocities of randomly sampled body frames, resulting from selected robot interactions, can be used to identify objects without any learned segmentation model.<n>We showcase the effectiveness of our proposed interactive perception method by achieving an average object segmentation accuracy rate 27.5% greater than state-of-the-art UOIS methods.
  arXiv  Detail & Related papers  (2025-07-14T20:02:52Z)
• EC-Flow: Enabling Versatile Robotic Manipulation from Action-Unlabeled Videos via Embodiment-Centric Flow [10.674192015199996]
  Embodiment-Centric Flow (EC-Flow) is a framework that learns manipulation from action-unlabeled videos.<n>Our key insight is that incorporating the embodiment's inherent kinematics significantly enhances generalization to versatile manipulation scenarios.<n> translating EC-Flow to executable robot actions only requires a standard robot URDF file to specify kinematic constraints.
  arXiv  Detail & Related papers  (2025-07-08T17:57:03Z)
• InterRVOS: Interaction-aware Referring Video Object Segmentation [44.55538737075162]
  We introduce Interaction-aware Referring Video Object (InterRVOS), a novel task that focuses on the modeling of interactions.<n>It requires the model to segment the actor and target objects separately, reflecting their asymmetric roles in an interaction.<n>We present InterRVOS-127K, a large-scale dataset with over 127K automatically annotated expressions, including interaction expressions annotated with distinct masks for actor and target objects.
  arXiv  Detail & Related papers  (2025-06-03T01:16:13Z)
• Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control [72.00655365269]
  We present RoboMaster, a novel framework that models inter-object dynamics through a collaborative trajectory formulation.<n>Unlike prior methods that decompose objects, our core is to decompose the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction.<n>Our method outperforms existing approaches, establishing new state-of-the-art performance in trajectory-controlled video generation for robotic manipulation.
  arXiv  Detail & Related papers  (2025-06-02T17:57:06Z)
• Keypoint Abstraction using Large Models for Object-Relative Imitation Learning [78.92043196054071]
  Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been proven effective as a succinct representation for essential object capturing features. We propose KALM, a framework that leverages large pre-trained vision-language models to automatically generate task-relevant and cross-instance consistent keypoints.
  arXiv  Detail & Related papers  (2024-10-30T17:37:31Z)
• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [57.942404069484134]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered.<n>Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.<n>We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)
• Learning Manipulation by Predicting Interaction [85.57297574510507]
  We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction. The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
  arXiv  Detail & Related papers  (2024-06-01T13:28:31Z)
• Learning Extrinsic Dexterity with Parameterized Manipulation Primitives [8.7221770019454]
  We learn a sequence of actions that utilize the environment to change the object's pose. Our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace.
  arXiv  Detail & Related papers  (2023-10-26T21:28:23Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.