TrackSSM: A General Motion Predictor by State-Space Model

• URL: http://arxiv.org/abs/2409.00487v2
• Date: Tue, 10 Sep 2024 07:41:08 GMT
• Title: TrackSSM: A General Motion Predictor by State-Space Model
• Abstract summary: TrackSSM is a unified encoder-decoder motion framework that uses data-dependent state space model to perform temporal motion of trajectories. Flow-SSM is a module that utilizes the position and motion information from historical trajectories to guide the temporal state transition of object bounding boxes. TrackSSM is applicable to various tracking scenarios and achieves excellent tracking performance across multiple benchmarks.
• Score: 14.733503542494674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal motion modeling has always been a key component in multiple object tracking (MOT) which can ensure smooth trajectory movement and provide accurate positional information to enhance association precision. However, current motion models struggle to be both efficient and effective across different application scenarios. To this end, we propose TrackSSM inspired by the recently popular state space models (SSM), a unified encoder-decoder motion framework that uses data-dependent state space model to perform temporal motion of trajectories. Specifically, we propose Flow-SSM, a module that utilizes the position and motion information from historical trajectories to guide the temporal state transition of object bounding boxes. Based on Flow-SSM, we design a flow decoder. It is composed of a cascaded motion decoding module employing Flow-SSM, which can use the encoded flow information to complete the temporal position prediction of trajectories. Additionally, we propose a Step-by-Step Linear (S$^2$L) training strategy. By performing linear interpolation between the positions of the object in the previous frame and the current frame, we construct the pseudo labels of step-by-step linear training, ensuring that the trajectory flow information can better guide the object bounding box in completing temporal transitions. TrackSSM utilizes a simple Mamba-Block to build a motion encoder for historical trajectories, forming a temporal motion model with an encoder-decoder structure in conjunction with the flow decoder. TrackSSM is applicable to various tracking scenarios and achieves excellent tracking performance across multiple benchmarks, further extending the potential of SSM-like temporal motion models in multi-object tracking tasks. Code and models are publicly available at \url{https://github.com/Xavier-Lin/TrackSSM}.

Related papers

• Following Motion for Sequential Modeling in Video Frame Interpolation [22.71397762551783]
  State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI)<n>We propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI.
  arXiv  Detail & Related papers  (2026-08-24T06:43:48Z)
• MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation [18.165045289069703]
  Motion-Aligned Selective Scan (MASS) is a novel framework that reformulates feature scanning from static spatial grids to dynamic motion trajectories.<n>MASS builds a feature sequence along each pixel's flow-guided trajectory and aggregates it with an SSM.
  arXiv  Detail & Related papers  (2026-06-26T04:56:31Z)
• BFMTrack: Latent Sequence Optimization for Physics-Based Motion Tracking with Behavioral Foundation Models [7.959692704349906]
  Behavioral Foundation Models (BFMs) organize a rich repertoire of physically plausible behaviors into a latent space, guided by a large-scale motion dataset.<n>These models excel at time-invariant tasks, such as goal-reaching and state-based reward optimization, but their latent space does not directly support time-varying objectives, such as tracking a motion sequence.<n>We propose a novel Latent Sequence Optimization (LSO) to address these shortcomings.
  arXiv  Detail & Related papers  (2026-06-23T18:12:22Z)
• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• SAMOFT: Robust Multi-Object Tracking via Region and Flow [42.6206516982406]
  Multi-object tracking (MOT) is a fundamental task in computer vision that requires continuously tracking multiple targets.<n>Most existing approaches rely on instance-level object features for trajectory association.<n>We propose SAMOFT, a robust tracker that leverages pixel-level cues to improve robustness under complex motion scenarios.
  arXiv  Detail & Related papers  (2026-05-10T08:39:54Z)
• SMTrack: State-Aware Mamba for Efficient Temporal Modeling in Visual Tracking [39.1131712751769]
  We propose a novel temporal modeling paradigm for visual tracking, termed State-aware Mamba Tracker (SMTrack)<n>SMTrack provides a neat pipeline for training and tracking without needing customized modules or substantial computational costs to build long-range temporal dependencies.<n>Extensive experimental results demonstrate that SMTrack achieves promising performance with low computational costs.
  arXiv  Detail & Related papers  (2026-02-02T05:44:59Z)
• MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing [53.98607267063729]
  MotionVerse is a framework to comprehend, generate, and edit human motion in both single-person and multi-person scenarios.<n>We employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens.<n>We also introduce a textitDelay Parallel Modeling strategy, which temporally staggers the encoding of residual token streams.
  arXiv  Detail & Related papers  (2025-09-28T04:20:56Z)
• TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion [18.191333256398845]
  A hurdle for event-based video frame (VFI) is how to deal with non-linear motion.<n>We propose a novel continuous point tracking-based VFI framework, named TimeTracker.<n>Our method outperforms prior arts in both motion estimation and frame quality.
  arXiv  Detail & Related papers  (2025-05-06T02:12:19Z)
• Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better [61.381599921020175]
  Temporal consistency is critical in video prediction to ensure that outputs are coherent and free of artifacts. Traditional methods, such as temporal attention and 3D convolution, may struggle with significant object motion. We propose the Tracktention Layer, a novel architectural component that explicitly integrates motion information using point tracks.
  arXiv  Detail & Related papers  (2025-03-25T17:58:48Z)
• Online Dense Point Tracking with Streaming Memory [54.22820729477756]
  Dense point tracking is a challenging task requiring the continuous tracking of every point in the initial frame throughout a substantial portion of a video. Recent point tracking algorithms usually depend on sliding windows for indirect information propagation from the first frame to the current one. We present a lightweight and fast model with textbfStreaming memory for dense textbfPOint textbfTracking and online video processing.
  arXiv  Detail & Related papers  (2025-03-09T06:16:49Z)
• Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking [53.33637391723555]
  We propose a unified multimodal spatial-temporal tracking approach named STTrack. In contrast to previous paradigms, we introduced a temporal state generator (TSG) that continuously generates a sequence of tokens containing multimodal temporal information. These temporal information tokens are used to guide the localization of the target in the next time state, establish long-range contextual relationships between video frames, and capture the temporal trajectory of the target.
  arXiv  Detail & Related papers  (2024-12-20T09:10:17Z)
• iMoT: Inertial Motion Transformer for Inertial Navigation [0.5199807441687141]
  iMoT is an innovative Transformer-based inertial odometry method. It retrieves cross-modal information from motion and rotation modalities for accurate positional estimation. iMoT significantly outperforms state-of-the-art methods in delivering superior robustness and accuracy in trajectory reconstruction.
  arXiv  Detail & Related papers  (2024-12-13T22:52:47Z)
• Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency [58.719310295870024]
  This paper presents an event-based framework for tracking any point. It tackles the challenges posed by spatial sparsity and motion sensitivity in events. It achieves 150% faster processing with competitive model parameters.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• MambaTrack: A Simple Baseline for Multiple Object Tracking with State Space Model [18.607106274732885]
  We introduce a Mamba-based motion model named Mamba moTion Predictor (MTP) MTP takes the spatial-temporal location dynamics of objects as input, captures the motion pattern using a bi-Mamba encoding layer, and predicts the next motion. Our proposed tracker, MambaTrack, demonstrates advanced performance on benchmarks such as Dancetrack and SportsMOT.
  arXiv  Detail & Related papers  (2024-08-17T11:58:47Z)
• Generalizable Implicit Motion Modeling for Video Frame Interpolation [51.966062283735596]
  Motion is critical in flow-based Video Frame Interpolation (VFI) We introduce General Implicit Motion Modeling (IMM), a novel and effective approach to motion modeling VFI. Our GIMM can be easily integrated with existing flow-based VFI works by supplying accurately modeled motion.
  arXiv  Detail & Related papers  (2024-07-11T17:13:15Z)
• ETTrack: Enhanced Temporal Motion Predictor for Multi-Object Tracking [4.250337979548885]
  We propose a motion-based MOT approach with an enhanced temporal motion predictor, ETTrack. Specifically, the motion predictor integrates a transformer model and a Temporal Convolutional Network (TCN) to capture short-term and long-term motion patterns. We show ETTrack achieves a competitive performance compared with state-of-the-art trackers on DanceTrack and SportsMOT.
  arXiv  Detail & Related papers  (2024-05-24T17:51:33Z)
• Exploring Learning-based Motion Models in Multi-Object Tracking [23.547018300192065]
  MambaTrack is an online motion-based tracker that outperforms all existing motion-based trackers on the challenging DanceTrack and SportsMOT datasets. We exploit the potential of the state-space-model in trajectory feature extraction to boost the tracking performance.
  arXiv  Detail & Related papers  (2024-03-16T06:26:52Z)
• Multi-step Temporal Modeling for UAV Tracking [14.687636301587045]
  We introduce MT-Track, a streamlined and efficient multi-step temporal modeling framework for enhanced UAV tracking. We unveil a unique temporal correlation module that dynamically assesses the interplay between the template and search region features. We propose a mutual transformer module to refine the correlation maps of historical and current frames by modeling the temporal knowledge in the tracking sequence.
  arXiv  Detail & Related papers  (2024-03-07T09:48:13Z)
• Delving into Motion-Aware Matching for Monocular 3D Object Tracking [81.68608983602581]
  We find that the motion cue of objects along different time frames is critical in 3D multi-object tracking. We propose MoMA-M3T, a framework that mainly consists of three motion-aware components. We conduct extensive experiments on the nuScenes and KITTI datasets to demonstrate our MoMA-M3T achieves competitive performance against state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-22T17:53:58Z)
• MotionTrack: Learning Motion Predictor for Multiple Object Tracking [68.68339102749358]
  We introduce a novel motion-based tracker, MotionTrack, centered around a learnable motion predictor. Our experimental results demonstrate that MotionTrack yields state-of-the-art performance on datasets such as Dancetrack and SportsMOT.
  arXiv  Detail & Related papers  (2023-06-05T04:24:11Z)
• MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking [56.92165669843006]
  We propose MotionTrack, which learns robust short-term and long-term motions in a unified framework to associate trajectories from a short to long range. For dense crowds, we design a novel Interaction Module to learn interaction-aware motions from short-term trajectories, which can estimate the complex movement of each target. For extreme occlusions, we build a novel Refind Module to learn reliable long-term motions from the target's history trajectory, which can link the interrupted trajectory with its corresponding detection.
  arXiv  Detail & Related papers  (2023-03-18T12:38:33Z)
• NEURAL MARIONETTE: A Transformer-based Multi-action Human Motion Synthesis System [51.43113919042621]
  We present a neural network-based system for long-term, multi-action human motion synthesis. The system can produce meaningful motions with smooth transitions from simple user input. We also present a new dataset dedicated to the multi-action motion synthesis task.
  arXiv  Detail & Related papers  (2022-09-27T07:10:20Z)
• Implicit Motion Handling for Video Camouflaged Object Detection [60.98467179649398]
  We propose a new video camouflaged object detection (VCOD) framework. It can exploit both short-term and long-term temporal consistency to detect camouflaged objects from video frames.
  arXiv  Detail & Related papers  (2022-03-14T17:55:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.