Segment Anything with Motion, Geometry, and Semantic Adaptation for Complex Nonlinear Visual Object Tracking

• URL: http://arxiv.org/abs/2605.22538v1
• Date: Thu, 21 May 2026 14:25:28 GMT
• Title: Segment Anything with Motion, Geometry, and Semantic Adaptation for Complex Nonlinear Visual Object Tracking
• Abstract summary: Recent vision foundation models, exemplified by SAM 2, learn strong video understanding priors from large-scale pretraining.<n>We propose SAMOSA, a new tracking framework that adapts SAM 2 to complex VOT scenarios by explicitly leveraging motion, geometry, and semantic cues.
• Score: 81.34609950921023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional visual object tracking (VOT) methods typically rely on task-specific supervised training, limiting their generalization to unseen objects and challenging scenarios with distractors, occlusion, and nonlinear motion. Recent vision foundation models, exemplified by SAM 2, learn strong video understanding priors from large-scale pretraining and offer a promising foundation for building more robust and generalizable trackers. However, directly applying SAM 2 to VOT remains suboptimal, as it does not explicitly model target motion dynamics or enforce geometric and semantic consistency across frames, both of which are essential for reliable tracking. To address this issue, we propose SAMOSA, a new tracking framework that adapts SAM 2 to complex VOT scenarios by explicitly leveraging motion, geometry, and semantic cues. Specifically, we introduce a lightweight nonlinear motion predictor to model target dynamics and guide mask selection as well as memory filtering. We further exploit semantic cues to detect target shifts and recover from tracking failures, while geometric cues are incorporated as structural constraints to improve tracking stability. In this way, SAMOSA bridges the gap between the implicit video understanding prior of SAM 2 and explicit tracking-oriented modeling. Extensive experiments show that SAMOSA consistently outperforms state-of-the-art SAM 2--based approaches on general benchmarks, demonstrates stronger generalization than supervised VOT methods, and achieves substantial gains on anti-UAV datasets, which typify complex nonlinear motion scenarios. Our code is available at https://github.com/DurYi/SAMOSA.

Related papers

• SUMO: Segment and Track Any Motion with Nonlinear State Space Models [20.136630716718383]
  Visual Object Tracking (VOT) and Moving Object (MOS) are two fundamental tasks in computer vision.<n>We propose SUMO, a zero-shot, training-free, unified framework integrating nonlinear dynamics with vision-based segmentation.<n>Specifically, we develop a nonlinear State Space Model (SSM) inspired by robotics principles to capture the complex object dynamics.
  arXiv  Detail & Related papers  (2026-06-29T06:55:13Z)
• Search2Motion: Training-Free Object-Level Motion Control via Attention-Consensus Search [11.772064059088821]
  We present Search2Motion, a training-free framework for object-level motion editing in image-to-video generation.<n>Unlike prior methods requiring trajectories, bounding boxes, masks, or motion fields, Search2Motion adopts target-frame-based control.<n>We show that Search2Motion consistently outperforms baselines on FLF2V-obj and VBench.
  arXiv  Detail & Related papers  (2026-03-17T16:02:38Z)
• SiamGM: Siamese Geometry-Aware and Motion-Guided Network for Real-Time Satellite Video Object Tracking [35.014174355605476]
  Single object tracking in satellite videos is inherently challenged by small target, blurred background, and large aspect ratio changes.<n>We propose SiamGM, a novel geometry-aware and motion-guided Siamese network to mitigate both spatial ambiguities and temporal information loss.<n>SiamGM introduces virtually no computational overhead, enabling real-time tracking at 130 frames per second (FPS)
  arXiv  Detail & Related papers  (2026-03-08T10:00:24Z)
• DMTrack: Deformable State-Space Modeling for UAV Multi-Object Tracking with Kalman Fusion and Uncertainty-Aware Association [18.68212724411998]
  Multi-object tracking (MOT) from unmanned aerial vehicles (UAVs) presents unique challenges due to unpredictable object motion.<n>We propose DMTrack, a deformable motion tracking framework tailored for UAV-based MOT.<n>Our method operates without appearance models and maintains competitive efficiency, highlighting its practicality for robust UAV-based tracking.
  arXiv  Detail & Related papers  (2025-10-15T13:54:25Z)
• Tracking the Unstable: Appearance-Guided Motion Modeling for Robust Multi-Object Tracking in UAV-Captured Videos [58.156141601478794]
  Multi-object tracking (UAVT) aims to track multiple objects while maintaining consistent identities across frames of a given video.<n>Existing methods typically model motion cues and appearance separately, overlooking their interplay and resulting in suboptimal tracking performance.<n>We propose AMOT, which exploits appearance and motion cues through two key components: an Appearance-Motion Consistency (AMC) matrix and a Motion-aware Track Continuation (MTC) module.
  arXiv  Detail & Related papers  (2025-08-03T12:06:47Z)
• SAMITE: Position Prompted SAM2 with Calibrated Memory for Visual Object Tracking [58.35852822355312]
  Visual Object Tracking (VOT) is widely used in applications like autonomous driving to continuously track targets in videos.<n>To address these issues, some methods propose to adapt the video foundation model SAM2 for VOT, where the tracking results of each frame would be encoded as memory for conditioning the rest of frames in an autoregressive manner.<n>We present a SAMITE model, built upon SAM2 with additional modules, to tackle these challenges.
  arXiv  Detail & Related papers  (2025-07-29T12:11:56Z)
• MotionTrack: Learning Motion Predictor for Multiple Object Tracking [68.68339102749358]
  We introduce a novel motion-based tracker, MotionTrack, centered around a learnable motion predictor. Our experimental results demonstrate that MotionTrack yields state-of-the-art performance on datasets such as Dancetrack and SportsMOT.
  arXiv  Detail & Related papers  (2023-06-05T04:24:11Z)
• SOOD: Towards Semi-Supervised Oriented Object Detection [57.05141794402972]
  This paper proposes a novel Semi-supervised Oriented Object Detection model, termed SOOD, built upon the mainstream pseudo-labeling framework. Our experiments show that when trained with the two proposed losses, SOOD surpasses the state-of-the-art SSOD methods under various settings on the DOTA-v1.5 benchmark.
  arXiv  Detail & Related papers  (2023-04-10T11:10:42Z)
• An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds [50.19288542498838]
  3D single object tracking in LiDAR point clouds (LiDAR SOT) plays a crucial role in autonomous driving. Current approaches all follow the Siamese paradigm based on appearance matching. We introduce a motion-centric paradigm to handle LiDAR SOT from a new perspective.
  arXiv  Detail & Related papers  (2023-03-21T17:28:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.