Triplet2Track: A Hierarchical System with Object-Centric Representations for Reliable Long-Horizon Manipulation

• URL: http://arxiv.org/abs/2608.22800v1
• Date: Mon, 24 Aug 2026 04:46:25 GMT
• Title: Triplet2Track: A Hierarchical System with Object-Centric Representations for Reliable Long-Horizon Manipulation
• Abstract summary: Triplet-to-Track System (TTS) is a closed-loop long-horizon imitation learning system that uses human videos to reduce reliance on robot-collected data.<n>TTS represents high-level subgoals as instance-grounded triplets, translates them into continuous track priors for execution, and monitors task progress from observations for online replanning.
• Score: 14.246688157481143
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring reliability in uncertain environments remains difficult for long-horizon robotic manipulation. End-to-end VLA models are data-heavy and opaque, making diagnosis and verification difficult. Hierarchical pipelines are more interpretable, but their plans are often weakly grounded in observations, weakly aligned with low-level actions, and computed without online feedback, leading to open-loop behavior and hallucinations. To address these issues, we introduce the Triplet-to-Track System (TTS), a closed-loop long-horizon imitation learning system that uses human videos to reduce reliance on robot-collected data. TTS represents high-level subgoals as instance-grounded triplets, translates them into continuous track priors for execution, and monitors task progress from observations for online replanning. Across diverse real-world long-horizon tasks, TTS achieves a 74.8\% average success rate and supports object-level and compositional generalization.

Related papers

• Compiling and Benchmarking Task-State Horizons for Embodied Agents [2.8397380902242966]
  Frontier agentic models are increasingly deployed as high-level planners for long-horizon embodied tasks.<n>We define the span of task-relevant state transitions that an agent must track as task-state horizon (TSH)<n>We introduce RoboGraph, a robotic task compiler that translates state-transition dependencies into executable symbolic graphs.
  arXiv  Detail & Related papers  (2026-08-08T09:45:01Z)
• Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation [40.0334846364089]
  Vision-Language-Action (VLA) models struggle with long-horizon tasks due to their Markovian nature-relying solely on current observations.<n>We introduce Cortex, a framework that conveys executable and tractable subtask plans from high-level VLM to low-level VLA.<n>This enables automatic annotation of over 4k hours of open-source video data and generation of 30 hours of simulation data.
  arXiv  Detail & Related papers  (2026-07-06T17:55:05Z)
• Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents [58.49879338545782]
  Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored.<n>We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model.<n>Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-06-22T09:32:28Z)
• GTA: Generating Long-Horizon Tasks for Web Agents at Scale [82.43869456830664]
  We introduce a scalable framework, GTA, that integrates crawling, retrieval-based seeding, in-context generation, and automated quality control.<n>We instantiate the pipeline on over 50 websites covering e-commerce, government, forums, and news, with multilingual and multi-hop coverage.<n>Our contributions are three-fold: (i) formalizing multi-hop web-agent task generation, (ii) proposing an efficient and validated pipeline for automatic data creation, and (iii) releasing a dynamic benchmark with reproducible evaluation.
  arXiv  Detail & Related papers  (2026-05-28T01:05:50Z)
• Long-Horizon Manipulation via Trace-Conditioned VLA Planning [55.80061850746898]
  LoHo-Manip is a framework that scales short-horizon VLA execution to long-horizon instruction following via a dedicated task-management VLM.<n>LoHo-Manip predicts a progress-aware remaining plan that combines a subtask sequence with an explicit done + remaining split as lightweight language memory.<n>The executor VLA is adapted to condition on the rendered trace, thereby turning long-horizon decision-making into repeated local control.
  arXiv  Detail & Related papers  (2026-04-23T17:59:04Z)
• Critic in the Loop: A Tri-System VLA Framework for Robust Long-Horizon Manipulation [5.339854280045898]
  Critic in the Loop is an adaptive hierarchical framework driven by dynamic VLM-Expert scheduling.<n>At its core is a bionic Tri-System architecture comprising a VLM brain for global reasoning, a VLA cerebellum for reactive execution, and a lightweight visual Critic.<n>Our architecture seamlessly integrates human-inspired rules to intuitively break infinite retry loops.
  arXiv  Detail & Related papers  (2026-03-05T13:55:33Z)
• V-CAGE: Context-Aware Generation and Verification for Scalable Long-Horizon Embodied Tasks [6.820118518027692]
  V-CAGE is a closed-loop framework for generating semantically aligned manipulation datasets at scale.<n>We propose a context-aware instantiation mechanism that enforces geometric consistency during scene synthesis.<n>We also employ a hierarchical instruction decomposition module to bridge the gap between abstract intent and low-level control.
  arXiv  Detail & Related papers  (2026-01-21T16:41:51Z)
• RoboTracer: Mastering Spatial Trace with Reasoning in Vision-Language Models for Robotics [53.053660003572965]
  We propose RoboTracer, a 3D-aware VLM that first achieves both 3D spatial referring and measuring.<n>RoboTracer advances multi-step metric-grounded reasoning via reinforcement fine-tuning.<n>We present TraceSpatial-Bench, a challenging benchmark to evaluate spatial tracing.
  arXiv  Detail & Related papers  (2025-12-15T18:52:43Z)
• Unsupervised Online 3D Instance Segmentation with Synthetic Sequences and Dynamic Loss [52.28880405119483]
  Unsupervised online 3D instance segmentation is a fundamental yet challenging task.<n>Existing methods, such as UNIT, have made progress in this direction but remain constrained by limited training diversity.<n>We propose a new framework that enriches the training distribution through synthetic point cloud sequence generation.
  arXiv  Detail & Related papers  (2025-09-27T08:53:27Z)
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection [56.66677293607114]
  We propose Code-as-Monitor (CaM) for both open-set reactive and proactive failure detection.<n>To enhance the accuracy and efficiency of monitoring, we introduce constraint elements that abstract constraint-related entities.<n>Experiments show that CaM achieves a 28.7% higher success rate and reduces execution time by 31.8% under severe disturbances.
  arXiv  Detail & Related papers  (2024-12-05T18:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.