HINT: Human-Intent Inception for Long-Horizon Robot Manipulation

• URL: http://arxiv.org/abs/2609.02653v1
• Date: Wed, 02 Sep 2026 14:26:30 GMT
• Title: HINT: Human-Intent Inception for Long-Horizon Robot Manipulation
• Abstract summary: We present HINT (Human-INTent INcepTion), an agentic framework inspired by the human manipulation principles.<n>We show that HINT substantially improves intent understanding, task progress, and end-to-end success across two foundation policies.
• Score: 11.55318573740128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans can perform complex manipulations given a simple intent through an overall instruction, while continuously adapting to evolving visual observations. However, current vision-language action (VLA) models and other action policies struggle to realize this high-level intelligent behavior under dense, evolving visual inputs and sparse language guidance. Visual correlations can then dominate semantic intent, leading actions to follow visual shortcuts rather than human goals. We present HINT (Human-INTent INcepTion), an agentic framework inspired by the human manipulation principles: semantic intent changes sparsely at manipulation-pattern transitions, whereas continuous control primarily depends on the evolving object-hand relationship. HINT invokes semantic reasoning only at pattern transitions to resolve the current subtask and target, then maintains this commitment through multi-view grounding and visual tracking. We explore two visual interfaces-image-space semantic highlighting and attention-prior injection-to communicate the tracked intent to the action policy without introducing additional trainable parameters into the foundation action model. Experiments across three long-horizon tasks and out-of-distribution variants show that HINT substantially improves intent understanding, task progress, and end-to-end success across two foundation policies while preserving low-latency control.

Related papers

• GIVE: Grounding Human Gestures in Vision-Language-Action Models [38.010995423052975]
  GIVE (Gesture Intent via Visual-Semantic Enhancement) is an effective approach to enhance pre-trained Vision-Language-Action (VLA) models with human gesture understanding.<n>In real-world HRI experiments, GIVE substantially outperforms the baseline, improving target object recognition accuracy by 40% and overall task success rate by 80%.
  arXiv  Detail & Related papers  (2026-06-11T14:59:38Z)
• Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulation [24.026684861369386]
  Vision-Language-Action (VLA) models have recently shown strong potential for robot learning by following language instructions.<n>It is difficult to describe which exact object to interact with among similar candidates, where to act on the object, or how the target may change during execution.<n>We propose Gaze2Act, a novel VLA framework that leverages human gaze as a dynamic and intuitive intent signal for complex interactive manipulation.
  arXiv  Detail & Related papers  (2026-05-28T17:37:16Z)
• Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models [15.486973209713954]
  GTA-VLA(Guide, Think, Act) is an interactive Vision-Language-Action framework.<n>It enables spatially steerable embodied reasoning by allowing users to guide robot policies with explicit visual cues.
  arXiv  Detail & Related papers  (2026-05-13T14:58:29Z)
• PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations [66.94988600664574]
  We present textbfPRTS (textbfPrimitive textbfReasoning and textbfTasking textbfSystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-04-30T06:14:02Z)
• Language-Grounded Decoupled Action Representation for Robotic Manipulation [78.42228162226839]
  We propose the Language-Grounded Decoupled Action Representation (LaDA) framework to connect perception and control.<n>LaDA introduces a fine-grained intermediate layer of three interpretable action primitives--translation, rotation, and gripper control--providing explicit semantic structure for low-level actions.<n>It further employs a semantic-guided soft-label contrastive learning objective to align similar action primitives across tasks, enhancing generalization and motion consistency.
  arXiv  Detail & Related papers  (2026-03-13T13:08:26Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• Intent at a Glance: Gaze-Guided Robotic Manipulation via Foundation Models [5.731055678884616]
  Eye gaze offers a fast, non-intrusive, and intent-rich input modality, making it an attractive channel for conveying user goals.<n>We present GAMMA, a system that leverages ego-centric gaze tracking and a vision-language model to infer user intent and autonomously execute robotic manipulation tasks.<n>Results demonstrate that GAMMA provides robust, intuitive, and generalizable control, highlighting the potential of combining foundation models and gaze for natural and scalable robot autonomy.
  arXiv  Detail & Related papers  (2026-01-08T19:33:03Z)
• Action-Sketcher: From Reasoning to Action via Visual Sketches for Long-Horizon Robotic Manipulation [37.76888706583038]
  Long-horizon robotic manipulation is increasingly important for real-world deployment.<n>Existing end-to-end and hierarchical Vision-Language-Action policies often rely on text-only cues.<n>We introduce Visual Sketch, an implausible visual intermediate that renders points, boxes, arrows, and typed relations in the robot's current views.<n>We present Action-Sketcher, a VLA framework that operates in a cyclic See-Think-Sketch-Act workflow.
  arXiv  Detail & Related papers  (2026-01-04T17:53:42Z)
• Mash, Spread, Slice! Learning to Manipulate Object States via Visual Spatial Progress [53.723881111373736]
  We present SPARTA, the first unified framework for the family of object state change manipulation tasks.<n>SPARTA integrates spatially progressing object change segmentation maps, a visual skill to perceive actionable vs. transformed regions, and dense rewards that capture incremental progress over time.<n>We validate SPARTA on a real robot for three challenging tasks across 10 diverse real-world objects.
  arXiv  Detail & Related papers  (2025-09-28T23:56:07Z)
• GoViG: Goal-Conditioned Visual Navigation Instruction Generation [69.79110149746506]
  We introduce Goal-Conditioned Visual Navigation Instruction Generation (GoViG), a new task that aims to autonomously generate precise and contextually coherent navigation instructions.<n>GoViG exclusively leverages raw egocentric visual data, substantially improving its adaptability to unseen and unstructured environments.
  arXiv  Detail & Related papers  (2025-08-13T07:05:17Z)
• Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation [52.6091162517921]
  INSIGHT is a two-stage framework for egocentric action anticipation.<n>In the first stage, INSIGHT focuses on extracting semantically rich features from hand-object interaction regions.<n>In the second stage, it introduces a reinforcement learning-based module that simulates explicit cognitive reasoning.
  arXiv  Detail & Related papers  (2025-08-03T12:52:27Z)
• Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning [69.71072181304066]
  We introduce Perceptive Dexterous Control (PDC), a framework for vision-driven whole-body control with simulated humanoids.<n>PDC operates solely on egocentric vision for task specification, enabling object search, target placement, and skill selection through visual cues.<n>We show that training from scratch with reinforcement learning can produce emergent behaviors such as active search.
  arXiv  Detail & Related papers  (2025-05-18T07:33:31Z)
• Learning Manipulation by Predicting Interaction [85.57297574510507]
  We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction. The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
  arXiv  Detail & Related papers  (2024-06-01T13:28:31Z)
• What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? [57.92924256181857]
  We find that visual representations designed for manipulation and control tasks do not necessarily generalize under subtle changes in lighting and scene texture. We find that emergent segmentation ability is a strong predictor of out-of-distribution generalization among ViT models.
  arXiv  Detail & Related papers  (2023-11-03T18:09:08Z)
• Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control [67.23580984118479]
  Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. Hand-eye Action Networks (HAN) can approximate human's hand-eye coordination behaviors by learning from human teleoperated demonstrations.
  arXiv  Detail & Related papers  (2021-02-28T01:49:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.