Rethinking Object-Centric Representations for Video Dynamics Modeling

• URL: http://arxiv.org/abs/2606.23436v1
• Date: Mon, 22 Jun 2026 14:55:41 GMT
• Title: Rethinking Object-Centric Representations for Video Dynamics Modeling
• Abstract summary: Unsupervised video object tracking aims to decompose dynamic scenes into persistent, object-centric entities without manual annotations.<n>Many recent approaches rely on slot-based representations, where a fixed set of latent variables ("slots") represent individual objects across frames.<n>We propose STAITUS, a unified framework that explicitly disentangles each slot into appearance and geometric pose (position/scale).
• Score: 17.157587526107598
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Unsupervised video object tracking aims to decompose dynamic scenes into persistent, object-centric entities without manual annotations. Many recent approaches rely on slot-based representations, where a fixed set of latent variables ("slots") represent individual objects across frames. To preserve object identity, these models enforce temporal consistency on slot embeddings. However, when appearance and pose are entangled, this consistency objective conflicts with object motion and viewpoint changes. As a result, slots tend to lock onto static regions (e.g., background) to satisfy the consistency objective, while foreground objects become fragmented across multiple slots or frequently swap identities. To address these limitations, we propose STAITUS, a unified framework that explicitly disentangles each slot into appearance and geometric pose (position/scale). Leveraging this disentanglement, STAITUS enforces within-frame spatial separation and applies temporal alignment only in appearance space, yielding sharper masks and more persistent identities under motion, occlusion, and object entry/exit. Furthermore, to mitigate over-segmentation, we introduce an adaptive gating mechanism that dynamically adjusts the number of active slots to match scene complexity. Extensive experiments on synthetic and real-world benchmarks demonstrate that STAITUS substantially outperforms state-of-the-art baselines in segmentation quality and tracking stability.

Related papers

• Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning [17.16289725621027]
  Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision.<n>Existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion.<n>We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that separates appearance from identity and reduces spurious updates from weakly matching slots.
  arXiv  Detail & Related papers  (2026-06-10T19:00:47Z)
• Controllable Video Object Insertion via Multiview Priors [30.363004603304418]
  Video object insertion is a critical task for dynamically inserting new objects into existing environments.<n>We propose a novel solution for Video Object Insertion, which integrates multi-view object priors.<n>We show that our solution significantly improves the quality of video object insertion, providing stable and realistic integration.
  arXiv  Detail & Related papers  (2026-04-16T02:39:15Z)
• AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation [55.94507360511886]
  We propose AnyCrowd, a Transformer-based video generation framework capable of scaling to an arbitrary number of characters.<n>Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement.<n>Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global-background coordination.
  arXiv  Detail & Related papers  (2026-03-16T15:25:04Z)
• GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injection [54.879037588415656]
  GenHOI is a lightweight augmentation to pretrained video generation models.<n>It injects reference-object information in a temporally balanced and spatially selective manner.<n>GenHOI significantly outperforms state-of-the-art HOI reenactment and all-in-one video editing methods.
  arXiv  Detail & Related papers  (2026-03-06T09:01:09Z)
• Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective [16.541717037293278]
  We introduce LIBERO-Mem, a non-Markovian task suite for stress-testing robotic manipulation under object-level partial observability.<n>It combines short- and long-horizon object tracking with temporally sequenced subgoals, requiring reasoning beyond the current frame.<n>We propose Embodied-SlotSSM, a slot-centric VLA framework built for temporal scalability.
  arXiv  Detail & Related papers  (2025-11-14T16:56:01Z)
• STD-GS: Exploring Frame-Event Interaction for SpatioTemporal-Disentangled Gaussian Splatting to Reconstruct High-Dynamic Scene [54.418259038624406]
  existing methods adopt unified representation model (e.g. Gaussian) to directly match scene from frame camera.<n>However, this unified paradigm fails in the potential temporal features of objects due to frame features and discontinuous spatial features between background and objects.<n>In this work, we introduce event camera to compensate for frame camera, and propose a distemporal-dentangle Gaussian splatting framework for high-dynamic scene reconstruction.
  arXiv  Detail & Related papers  (2025-06-29T09:32:06Z)
• Spatio-temporal Graph Learning on Adaptive Mined Key Frames for High-performance Multi-Object Tracking [5.746443489229576]
  Key Frame Extraction (KFE) module leverages reinforcement learning to adaptively segment videos.<n> Intra-Frame Feature Fusion (IFF) module uses a Graph Convolutional Network (GCN) to facilitate information exchange between the target and surrounding objects.<n>Our proposed tracker achieves impressive results on the MOT17 dataset.
  arXiv  Detail & Related papers  (2025-01-17T11:36:38Z)
• ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives [109.11714588441511]
  The Ego-Exo object correspondence task aims to understand object relations across ego-exo perspectives through segmentation.<n> PSALM, a recently proposed segmentation method, stands out as a notable exception with its demonstrated zero-shot ability on this task.<n>We propose ObjectRelator, a novel approach featuring two key modules: Multimodal Condition Fusion and SSL-based Cross-View Object Alignment.
  arXiv  Detail & Related papers  (2024-11-28T12:01:03Z)
• ZoomNeXt: A Unified Collaborative Pyramid Network for Camouflaged Object Detection [70.11264880907652]
  Recent object (COD) attempts to segment objects visually blended into their surroundings, which is extremely complex and difficult in real-world scenarios. We propose an effective unified collaborative pyramid network that mimics human behavior when observing vague images and camouflaged zooming in and out. Our framework consistently outperforms existing state-of-the-art methods in image and video COD benchmarks.
  arXiv  Detail & Related papers  (2023-10-31T06:11:23Z)
• Object-Centric Multiple Object Tracking [124.30650395969126]
  This paper proposes a video object-centric model for multiple-object tracking pipelines. It consists of an index-merge module that adapts the object-centric slots into detection outputs and an object memory module. Benefited from object-centric learning, we only require sparse detection labels for object localization and feature binding.
  arXiv  Detail & Related papers  (2023-09-01T03:34:12Z)
• Identity-Consistent Aggregation for Video Object Detection [21.295859014601334]
  In Video Object Detection (VID), a common practice is to leverage the rich temporal contexts from the video to enhance the object representations in each frame. We propose ClipVID, a VID model equipped with Identity-Consistent Aggregation layers specifically designed for mining fine-grained and identity-consistent temporal contexts. Experiments demonstrate the superiority of our method: a state-of-the-art (SOTA) performance (84.7% mAP) on the ImageNet VID dataset while running at a speed about 7x faster (39.3 fps) than previous SOTAs.
  arXiv  Detail & Related papers  (2023-08-15T12:30:22Z)
• SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition [69.90530987240899]
  We present an unsupervised variational approach to this problem. Our model learns to infer two sets of latent representations from RGB video input alone. It represents object attributes in an allocentric manner which does not depend on viewpoint.
  arXiv  Detail & Related papers  (2021-06-07T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.