Unified Learning of Temporal Task Structure and Action Timing for Bimanual Robot Manipulation

• URL: http://arxiv.org/abs/2603.06538v1
• Date: Fri, 06 Mar 2026 18:25:42 GMT
• Title: Unified Learning of Temporal Task Structure and Action Timing for Bimanual Robot Manipulation
• Abstract summary: Temporal task structure is fundamental for bimanual manipulation.<n>Existing approaches address these two levels in isolation.<n>This work presents an approach for learning both symbolic and subsymbolic temporal task constraints from human demonstrations.
• Score: 11.051601535345819
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal task structure is fundamental for bimanual manipulation: a robot must not only know that one action precedes or overlaps another, but also when each action should occur and how long it should take. While symbolic temporal relations enable high-level reasoning about task structure and alternative execution sequences, concrete timing parameters are equally essential for coordinating two hands at the execution level. Existing approaches address these two levels in isolation, leaving a gap between high-level task planning and low-level movement synchronization. This work presents an approach for learning both symbolic and subsymbolic temporal task constraints from human demonstrations and deriving executable, temporally parametrized plans for bimanual manipulation. Our contributions are (i) a 3-dimensional representation of timings between two actions with methods based on multivariate Gaussian Mixture Models to represent temporal relationships between actions on a subsymbolic level, (ii) a method based on the Davis-Putnam-Logemann-Loveland (DPLL) algorithm that finds and ranks all contradiction-free assignments of Allen relations to action pairs, representing different modes of a task, and (iii) an optimization-based planning system that combines the identified symbolic and subsymbolic temporal task constraints to derive temporally parametrized plans for robot execution. We evaluate our approach on several datasets, demonstrating that our method generates temporally parametrized plans closer to human demonstrations than the most characteristic demonstration baseline.

Related papers

• EnergyAction: Unimanual to Bimanual Composition with Energy-Based Models [73.21721376238233]
  EnergyAction is a novel framework that compositionally transfers unimanual manipulation policies to bimanual tasks.<n>First, we model individual unimanual policies as EBMs and leverage their compositional properties to compose left and right arm actions.<n>Second, we introduce an energy-based temporal-spatial coordination mechanism through energy constraints.<n>Third, we propose two different energy-aware denoising strategies that dynamically adapt denoising steps based on action quality assessment.
  arXiv  Detail & Related papers  (2026-03-09T04:50:17Z)
• BiKC+: Bimanual Hierarchical Imitation with Keypose-Conditioned Coordination-Aware Consistency Policies [25.25845809170421]
  We propose a novel keypose-conditioned coordination-aware consistency policy tailored for bimanual manipulation.<n>Our framework instantiates hierarchical imitation learning with a high-level keypose predictor and a low-level trajectory generator.<n>In particular, we devise an innovative approach for identifying bimanual keyposes, considering both robot-centric action features and task-centric operation styles.
  arXiv  Detail & Related papers  (2026-01-17T17:33:18Z)
• Hierarchical Relation-augmented Representation Generalization for Few-shot Action Recognition [43.84348967231349]
  Few-shot action recognition aims to recognize novel action categories with few exemplars.<n>Existing methods typically learn frame-level representations for each video by designing inter-frame temporal modeling strategies.<n>We propose HR2G-shot, a Hierarchical Relation-augmented Representation Generalization framework for FSAR.
  arXiv  Detail & Related papers  (2025-04-14T10:23:22Z)
• A Task and Motion Planning Framework Using Iteratively Deepened AND/OR Graph Networks [3.635602838654497]
  We present an approach for integrated task and motion planning based on an AND/OR graph network.<n>We leverage it to implement different classes of task and motion planning problems (TAMP)<n>The approach is evaluated and validated both in simulation and with a real dual-arm robot manipulator, that is, Baxter from Rethink Robotics.
  arXiv  Detail & Related papers  (2025-03-10T17:28:22Z)
• BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulation [48.08416841005715]
  We introduce a novel keypose-conditioned consistency policy tailored for bimanual manipulation. It is a hierarchical imitation learning framework that consists of a high-level keypose predictor and a low-level trajectory generator. Simulated and real-world experimental results demonstrate that the proposed approach surpasses baseline methods in terms of success rate and operational efficiency.
  arXiv  Detail & Related papers  (2024-06-14T14:49:12Z)
• Temporal Planning via Interval Logic Satisfiability for Autonomous Systems [0.0]
  We consider formulations of temporal planning where intervals are associated with both action and fluent atoms and relations between these are given as sentences in Allen's Interval Logic. We propose a notion of planning graphs that can account for complex relations between actions and fluents as a Constraint Programming (CP) model. We demonstrate our algorithm outperforms existing PDDL 2.1 planners in the case studies.
  arXiv  Detail & Related papers  (2024-06-14T02:21:53Z)
• Unified Task and Motion Planning using Object-centric Abstractions of Motion Constraints [56.283944756315066]
  We propose an alternative TAMP approach that unifies task and motion planning into a single search. Our approach is based on an object-centric abstraction of motion constraints that permits leveraging the computational efficiency of off-the-shelf AI search to yield physically feasible plans.
  arXiv  Detail & Related papers  (2023-12-29T14:00:20Z)
• Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning [85.84504287685884]
  Skip-Plan is a condensed action space learning method for procedure planning in instructional videos. By skipping uncertain nodes and edges in action chains, we transfer long and complex sequence functions into short but reliable ones. Our model explores all sorts of reliable sub-relations within an action sequence in the condensed action space.
  arXiv  Detail & Related papers  (2023-10-01T08:02:33Z)
• Decomposition-based Hierarchical Task Allocation and Planning for Multi-Robots under Hierarchical Temporal Logic Specifications [9.150196865878234]
  We formulate a decomposition-based hierarchical framework for robotic planning with temporal logic specifications. A Mixed Linear Program is used to assign sub-tasks to various robots. Our approach was experimentally applied to domains of navigation and manipulation.
  arXiv  Detail & Related papers  (2023-08-20T23:53:13Z)
• Optimal task and motion planning and execution for human-robot multi-agent systems in dynamic environments [54.39292848359306]
  We propose a combined task and motion planning approach to optimize sequencing, assignment, and execution of tasks. The framework relies on decoupling tasks and actions, where an action is one possible geometric realization of a symbolic task. We demonstrate the approach effectiveness in a collaborative manufacturing scenario, in which a robotic arm and a human worker shall assemble a mosaic.
  arXiv  Detail & Related papers  (2023-03-27T01:50:45Z)
• Learning Sequence Representations by Non-local Recurrent Neural Memory [61.65105481899744]
  We propose a Non-local Recurrent Neural Memory (NRNM) for supervised sequence representation learning. Our model is able to capture long-range dependencies and latent high-level features can be distilled by our model. Our model compares favorably against other state-of-the-art methods specifically designed for each of these sequence applications.
  arXiv  Detail & Related papers  (2022-07-20T07:26:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.