AdaTracker: Learning Adaptive In-Context Policy for Cross-Embodiment Active Visual Tracking

• URL: http://arxiv.org/abs/2604.20305v1
• Date: Wed, 22 Apr 2026 08:06:32 GMT
• Title: AdaTracker: Learning Adaptive In-Context Policy for Cross-Embodiment Active Visual Tracking
• Abstract summary: Existing approaches typically train separate models for each embodiment, leading to poor scalability and limited generalization.<n>AdaTracker is an adaptive in-context policy learning framework that robustly tracks targets on diverse robot morphologies.<n> Experiments in both simulation and the real world demonstrate that AdaTracker significantly outperforms state-of-the-art methods in cross-embodiment generalization, sample efficiency, and zero-shot adaptation.
• Score: 42.310377711916104
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Realizing active visual tracking with a single unified model across diverse robots is challenging, as the physical constraints and motion dynamics vary drastically from one platform to another. Existing approaches typically train separate models for each embodiment, leading to poor scalability and limited generalization. To address this, we propose AdaTracker, an adaptive in-context policy learning framework that robustly tracks targets on diverse robot morphologies. Our key insight is to explicitly model embodiment-specific constraints through an Embodiment Context Encoder, which infers embodiment-specific constraints from history. This contextual representation dynamically modulates a Context-Aware Policy, enabling it to infer optimal control actions for unseen embodiments in a zero-shot manner. To enhance robustness, we introduce two auxiliary objectives to ensure accurate context identification and temporal consistency. Experiments in both simulation and the real world demonstrate that AdaTracker significantly outperforms state-of-the-art methods in cross-embodiment generalization, sample efficiency, and zero-shot adaptation.

Related papers

• AnyTrack: Unifying Visual Object Tracking with Any Modalities [71.788116599779]
  We propose a unified framework called AnyTrack for object tracking with any modalities.<n> Modality-aware Interaction Module (MIM) to facilitate dynamic interaction across diverse modalities.<n> Context Understanding Module (CUM) to establish spatial correspondence between visual and target locations.
  arXiv  Detail & Related papers  (2026-08-07T03:47:34Z)
• DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation [18.601805553878638]
  DyPES-VLA is a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control.<n>As a generalist policy, our ourmethod achieves state-of-the-art performance across simulation and real-world evaluations.
  arXiv  Detail & Related papers  (2026-08-06T17:59:20Z)
• GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking [71.9852231453445]
  Event cameras enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion.<n>Existing event-based feature tracking methods rely on fixed rules based on hand-tuning for event accumulation.<n>We introduce reinforcement learning framework to adaptively control the accumulation process for online event-based feature tracking.
  arXiv  Detail & Related papers  (2026-07-17T07:18:07Z)
• KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation [2.532260218883922]
  Generalizing manipulation policies across robot embodiments remains difficult because standard policies entangle task reasoning with embodiment-specific motor control.<n>We introduce Kinematic Interaction Transfer across Embodiments (KITE), which decouples manipulation into embodiment-agnostic task reasoning and embodiment-specific motor control.<n>KITE consistently achieves zero-shot transfer to structurally different target embodiments, outperforming state-of-the-art baselines in transfer success and task-embodiment scope.
  arXiv  Detail & Related papers  (2026-06-20T15:51:30Z)
• MOTIF: Learning Action Motifs for Few-shot Cross-Embodiment Transfer [55.982504915794514]
  Cross-embodiment policies typically rely on shared-private architectures.<n>We introduce MOTIF for efficient few-shot cross-embodiment transfer.<n>We show that MOTIF significantly outperforms strong baselines in few-shot transfer scenarios.
  arXiv  Detail & Related papers  (2026-02-14T13:21:40Z)
• AdaMorph: Unified Motion Retargeting via Embodiment-Aware Adaptive Transformers [49.796014934503184]
  We present a unified neural framework that enables a single model to adapt human motion to diverse robot morphologies.<n> Experimental results on 12 distinct humanoid robots demonstrate that AdaMorph effectively control across heterogeneous topologies.
  arXiv  Detail & Related papers  (2026-01-12T07:39:38Z)
• Joint Flow Trajectory Optimization For Feasible Robot Motion Generation from Video Demonstrations [8.133207162076877]
  We propose a framework for grasp pose generation and object trajectory imitation under the video-based Learning-from-Demonstration (LfD) paradigm.<n>Rather than directly imitating human hand motions, our method treats demonstrations as object-centric guides.<n>We validate our approach in both simulation and real-world experiments across diverse real-world manipulation tasks.
  arXiv  Detail & Related papers  (2025-09-25T03:11:07Z)
• TrackVLA: Embodied Visual Tracking in the Wild [34.03604806748204]
  Embodied visual tracking is a fundamental skill in Embodied AI, enabling an agent to follow a specific target in dynamic environments using only egocentric vision.<n>Existing approaches typically address this challenge through a modular separation of recognition and planning.<n>We propose TrackVLA, a Vision-Language-Action model that learns the synergy between object recognition and trajectory planning.
  arXiv  Detail & Related papers  (2025-05-29T07:28:09Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• Diffusion Transformer Policy [48.50988753948537]
  We propose a large multi-modal diffusion transformer, dubbed as Diffusion Transformer Policy, to model continuous end-effector actions.<n>By leveraging the scaling capability of transformers, the proposed approach can effectively model continuous end-effector actions across large diverse robot datasets.
  arXiv  Detail & Related papers  (2024-10-21T12:43:54Z)
• ReCoRe: Regularized Contrastive Representation Learning of World Model [21.29132219042405]
  We present a world model that learns invariant features using contrastive unsupervised learning and an intervention-invariant regularizer. Our method outperforms current state-of-the-art model-based and model-free RL methods and significantly improves on out-of-distribution point navigation tasks evaluated on the iGibson benchmark.
  arXiv  Detail & Related papers  (2023-12-14T15:53:07Z)
• Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments [3.5309638744466167]
  This work developed a meta-learning approach that adapts the control policy on the fly to different changing conditions for robust locomotion. The proposed method constantly updates the interaction model, samples feasible sequences of actions of estimated the state-action trajectories, and then applies the optimal actions to maximize the reward.
  arXiv  Detail & Related papers  (2021-01-19T12:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.