ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control

• URL: http://arxiv.org/abs/2606.03177v1
• Date: Tue, 02 Jun 2026 05:31:32 GMT
• Title: ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control
• Abstract summary: We introduce ConTrack, a reinforcement learning framework that scales with tracking data.<n>ConTrack treats object tracking as a constraint and allocates remaining control authority to motion fidelity.<n>In addition, ConTrack stabilizes long-horizon learning with an adaptive mid-trajectory reset library.
• Score: 8.827966532956918
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human demonstrations provide strong priors for robot manipulation, yet it is non-trivial to transfer them to execute on real robots due to the kinematic gap. In dexterous manipulation, it remains challenging to track long-horizon, contact-rich sequences even in simulators: a reference-tracking policy must keep objects on their target trajectories while preserving demonstrated joint motion and contact timing. Existing approaches often rely on hand-crafted reward tuning that require per-sequence tuning and break under limited interaction budgets. We introduce ConTrack, a reinforcement learning (RL) framework that scales with tracking data. ConTrack treats object tracking as a constraint and allocates remaining control authority to motion fidelity, which allows it to adapt task--style trade-offs online using a dual-variable update. In addition, ConTrack also stabilizes long-horizon learning with an adaptive mid-trajectory reset library that reuses policy-reachable simulator states. Our qualitative and quantitative results in simulation tracking and real robot demonstrate that ConTrack improves success and object pose accuracy significantly over prior arts while preserving joint and contact fidelity. Website: https://www.lyt0112.com/projects/ConTrack.

Related papers

• Robotic Servo Tracking of Moving Targets with Dynamic Imitation Constraints [5.415143520107641]
  This paper proposes a servo tracking method based on imitation trajectory constraints.<n>A time-scalable deformation mechanism and a trajectory modulation incorporating shape and amplitude components are introduced to generate a series of trajectories in real time.<n>The proposed method can achieve dynamic obstacle avoidance and high-precision convergence compared with several state-of-the-art methods in complex environments.
  arXiv  Detail & Related papers  (2026-09-13T15:17:44Z)
• GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking [71.9852231453445]
  Event cameras enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion.<n>Existing event-based feature tracking methods rely on fixed rules based on hand-tuning for event accumulation.<n>We introduce reinforcement learning framework to adaptively control the accumulation process for online event-based feature tracking.
  arXiv  Detail & Related papers  (2026-07-17T07:18:07Z)
• Load-Aware Locomotion Control for Humanoid Robots in Industrial Transportation Tasks [19.55626538861657]
  This paper presents a load-aware locomotion framework for industrial humanoids based on a decoupled yet coordinated loco-manipulation architecture.<n>Lower-body locomotion is controlled via a reinforcement learning policy producing residual joint actions on kinematically derived nominal configurations.<n>The framework is trained entirely in simulation and deployed on a full-size humanoid robot without fine-tuning.
  arXiv  Detail & Related papers  (2026-03-15T09:57:32Z)
• OmniTrack: General Motion Tracking via Physics-Consistent Reference [54.256312586493415]
  We introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking.<n>Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions.<n>In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels.
  arXiv  Detail & Related papers  (2026-02-27T09:12:15Z)
• Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds [20.44610499489492]
  Bipedal humanoid robots must coordinate balance, timing, and contact decisions when locomoting on constrained footholds.<n>We introduce a locomotion framework in which a reduced-order stepping planner supplies dynamically consistent motion targets.<n>This combination of structured footstep planning and data-driven adaptation produces accurate, agile, and hardware-validated stepping-stone locomotion on a humanoid robot.
  arXiv  Detail & Related papers  (2026-01-09T19:56:42Z)
• CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation [70.5382178207975]
  hIsight Perturbation (CHIP) is a plug-and-play module that enables controllable end-effector stiffness.<n>CHIP is easy to implement and requires neither data augmentation nor additional reward tuning.<n>We show that a generalist motion-tracking controller trained with CHIP can perform a diverse set of forceful manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-16T18:56:04Z)
• KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills [58.73043119128804]
  This paper presents a physics-based humanoid control framework, aiming to master highly-dynamic human behaviors such as Kungfu and dancing.<n>For motion processing, we design a pipeline to extract, filter out, correct, and retarget motions, while ensuring compliance with physical constraints.<n>For motion imitation, we formulate a bi-level optimization problem to dynamically adjust the tracking accuracy tolerance.<n>In experiments, we train whole-body control policies to imitate a set of highly-dynamic motions.
  arXiv  Detail & Related papers  (2025-06-15T13:58:53Z)
• DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References [18.947295547196774]
  We address the challenge of developing a generalizable neural tracking controller for dexterous manipulation from human references.<n>We introduce an approach that curates large-scale successful robot tracking demonstrations.<n>Our method achieves over a 10% improvement in success rates compared to leading baselines.
  arXiv  Detail & Related papers  (2025-02-13T18:59:13Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• STT: Stateful Tracking with Transformers for Autonomous Driving [48.621552393062686]
  Tracking objects in three-dimensional space is critical for autonomous driving. To ensure safety while driving, the tracker must be able to reliably track objects across frames and accurately estimate their states such as velocity and acceleration in the present. We propose STT, a Stateful Tracking model built with Transformers, that can consistently track objects in the scenes while also predicting their states accurately.
  arXiv  Detail & Related papers  (2024-04-30T23:04:36Z)
• TLControl: Trajectory and Language Control for Human Motion Synthesis [68.09806223962323]
  We present TLControl, a novel method for realistic human motion synthesis. It incorporates both low-level Trajectory and high-level Language semantics controls. It is practical for interactive and high-quality animation generation.
  arXiv  Detail & Related papers  (2023-11-28T18:54:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.