MSACT: Multistage Spatial Alignment for Stable Low-Latency Fine Manipulation

• URL: http://arxiv.org/abs/2605.00475v1
• Date: Fri, 01 May 2026 07:35:15 GMT
• Title: MSACT: Multistage Spatial Alignment for Stable Low-Latency Fine Manipulation
• Abstract summary: Real-world fine manipulation, particularly in bimanual manipulation, requires low-latency control and stable visual localization.<n>Existing approaches make different trade-offs: action-chunking policies such as ACT enable low-latency execution and data efficiency but rely on dense visual features without explicit spatial consistency.<n>We introduce a multistage spatial attention module that extracts stable 2D attention points and jointly predicts future attention sequences with a temporal alignment loss.
• Score: 4.439585594082787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world fine manipulation, particularly in bimanual manipulation, typically requires low-latency control and stable visual localization, while collecting large-scale data is costly and limited demonstrations may lead to localization drift. Existing approaches make different trade-offs: action-chunking policies such as ACT enable low-latency execution and data efficiency but rely on dense visual features without explicit spatial consistency, generative methods such as Diffusion Policy improve expressiveness but can incur iterative sampling latency, vision-language-action and voxel-based methods enhance generalization and geometric grounding but require higher computational cost and system complexity. We introduce a multistage spatial attention module that extracts stable 2D attention points and jointly predicts future attention sequences with a temporal alignment loss. Built upon ACT with a pretrained ResNet visual prior, a multistage attention module extracts task-relevant 2D attention points as a local spatial modality for action prediction. To maintain consistent object tracking, we introduce a self-supervised objective that aligns predicted attention sequences with visual features from future frames, suppressing drift without keypoint annotations and improving stability of the vision-to-action mapping under limited data. Experiments on simulated and real-world fine manipulation tasks, conducted on the ALOHA bimanual platform, evaluate task success, attention drift, inference latency, and robustness to visual disturbances. Results indicate improvements in localization stability and task performance while maintaining low-latency inference under the tested conditions.

Related papers

• ARP: Enhancing Quantized Skill Abstractions via Visual Alignment and Iterative Refinement for Robotic Manipulation [12.782646427089967]
  Aligned Refinement Policy (ARP) couples semantic grounding with execution-level refinement.<n>ARP achieves state-of-the-art performance on the LIBERO and Meta-World benchmarks.
  arXiv  Detail & Related papers  (2026-06-21T12:49:46Z)
• Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration [7.647944993041615]
  We introduce a training-free inference strategy to dynamically correct semantic drift without relying on abrupt truncations.<n>By computing an outlier-resistant statistical midpoint across various attention heads, we establish a stable anchor for reliable visual representations.<n>This recalibration process effectively rectifies visual-semantic misalignments while fully preserving generative fluency and language priors.
  arXiv  Detail & Related papers  (2026-05-24T09:14:49Z)
• Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention [20.653155039432463]
  Event cameras provide sequential visual data with spatial sparsity and high temporal resolution, making them attractive for low-latency object detection.<n>Existing asynchronous event-based neural networks realize this low-latency advantage by updating predictions event-by-event, but still suffer from two bottlenecks: recurrent architectures are difficult to train efficiently on long sequences, and improving accuracy often increases per-event computation and latency.<n>We propose Spatially-Sparse Linear Attention (SSLA), which introduces a mixture-of-spaces state decomposition and a scatter-compute-gather training procedure, enabling state-level sparsity as well as
  arXiv  Detail & Related papers  (2026-03-06T12:44:00Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention [92.85371254435074]
  PosA-VLA framework anchors visual attention via pose-conditioned supervision, consistently guiding the model's perception toward task-relevant regions.<n>We show that our method executes embodied tasks with precise and time-efficient behavior across diverse robotic manipulation benchmarks.
  arXiv  Detail & Related papers  (2025-12-03T12:14:29Z)
• UTAL-GNN: Unsupervised Temporal Action Localization using Graph Neural Networks [0.0]
  Fine-grained action localization in untrimmed sports videos presents a significant challenge due to rapid and subtle motion transitions.<n>Existing supervised and weakly supervised solutions often rely on extensive datasets and high-capacity models, making them computationally intensive and less adaptable to real-world scenarios.<n>Our approach pre-trains an Attention-based Spatio-Temporal Graph Convolutional Network (ASTGCN) on a pose-sequence denoising annotated with blockwise partitions.<n>Our method achieves a mean Average Precision (mAP) of 82.66% and average latency localization of 29.09 ms on the DSV Diving dataset
  arXiv  Detail & Related papers  (2025-08-27T07:51:02Z)
• Inference-Time Gaze Refinement for Micro-Expression Recognition: Enhancing Event-Based Eye Tracking with Motion-Aware Post-Processing [2.5465367830324905]
  Event-based eye tracking holds significant promise for fine-grained cognitive state inference.<n>We introduce a model-agnostic, inference-time refinement framework to enhance the output of existing event-based gaze estimation models.
  arXiv  Detail & Related papers  (2025-06-14T14:48:11Z)
• SuperFlow++: Enhanced Spatiotemporal Consistency for Cross-Modal Data Pretraining [62.433137130087445]
  SuperFlow++ is a novel framework that integrates pretraining and downstream tasks using consecutive camera pairs.<n>We show that SuperFlow++ outperforms state-of-the-art methods across diverse tasks and driving conditions.<n>With strong generalizability and computational efficiency, SuperFlow++ establishes a new benchmark for data-efficient LiDAR-based perception in autonomous driving.
  arXiv  Detail & Related papers  (2025-03-25T17:59:57Z)
• LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry [53.5449912019877]
  We present the Long-term Effective Any Point Tracking (LEAP) module.<n>LEAP innovatively combines visual, inter-track, and temporal cues with mindfully selected anchors for dynamic track estimation.<n>Based on these traits, we develop LEAP-VO, a robust visual odometry system adept at handling occlusions and dynamic scenes.
  arXiv  Detail & Related papers  (2024-01-03T18:57:27Z)
• Layout Sequence Prediction From Noisy Mobile Modality [53.49649231056857]
  Trajectory prediction plays a vital role in understanding pedestrian movement for applications such as autonomous driving and robotics. Current trajectory prediction models depend on long, complete, and accurately observed sequences from visual modalities. We propose LTrajDiff, a novel approach that treats objects obstructed or out of sight as equally important as those with fully visible trajectories.
  arXiv  Detail & Related papers  (2023-10-09T20:32:49Z)
• Progressive Self-Guided Loss for Salient Object Detection [102.35488902433896]
  We present a progressive self-guided loss function to facilitate deep learning-based salient object detection in images. Our framework takes advantage of adaptively aggregated multi-scale features to locate and detect salient objects effectively.
  arXiv  Detail & Related papers  (2021-01-07T07:33:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.