Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning

• URL: http://arxiv.org/abs/2606.12601v1
• Date: Wed, 10 Jun 2026 19:00:47 GMT
• Title: Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning
• Abstract summary: Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision.<n>Existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion.<n>We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that separates appearance from identity and reduces spurious updates from weakly matching slots.
• Score: 17.16289725621027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision. However, existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion. First, they typically encode both the per-frame appearance of an object and its identity across frames in a single slot vector, creating an objective conflict that leads to slot swapping: reconstruction requires sensitivity to transient visual changes, whereas temporal consistency requires invariance to them. Second, the token renormalization used in Slot Attention can amplify weakly attending slots, allowing them to absorb tokens from other objects and destabilize slot-to-object correspondence. We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that addresses these limitations by separating appearance from identity and by reducing spurious updates from weakly matching slots. DSSA decomposes each slot into a local state for per-frame appearance and an identity state for temporally stable object information, thereby aligning reconstruction and temporal consistency with separate representations. The identity state is updated through a learned recurrent transition that acts as a temporal filter on the local state, while competition-modulated aggregation (CMA) down-weights updates from weakly matching slots and prevents them from absorbing tokens from other objects. Experiments on MOVi-C, MOVi-D, and YouTube-VIS demonstrate that DSSA consistently improves segmentation quality and temporal consistency over prior methods, while also yielding stronger downstream object recognition and video dynamics prediction. Code and models will be made publicly available upon acceptance.

Related papers

• Rethinking Object-Centric Representations for Video Dynamics Modeling [17.157587526107598]
  Unsupervised video object tracking aims to decompose dynamic scenes into persistent, object-centric entities without manual annotations.<n>Many recent approaches rely on slot-based representations, where a fixed set of latent variables ("slots") represent individual objects across frames.<n>We propose STAITUS, a unified framework that explicitly disentangles each slot into appearance and geometric pose (position/scale).
  arXiv  Detail & Related papers  (2026-06-22T14:55:41Z)
• TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation [28.696788831100644]
  Unsupervised video object-centric learning aims to decompose dynamic scenes into temporally persistent entity representations.<n>We propose Temporal Slot Activation (TSA), a mechanism that learns a per-slot, per-frame activation score without visibility supervision.<n>TSA consistently improves object decomposition and temporal identity preservation, with large gains on long, heavily occluded videos.
  arXiv  Detail & Related papers  (2026-06-10T20:54:45Z)
• Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation [79.94088803584262]
  Identity-preserving video generation (IPVG) aims to synthesize high-fidelity videos that follow text prompts while preserving a reference identity.<n>We propose ST-DRC, an effective Spatial-Temporal Decoupled Reference Conditioning framework for identity-preserving text-to-video generation.<n>We show that ST-DRC achieves strong identity preservation, prompt alignment, temporal consistency, and video quality with a lightweight design built on LTX-2.3.
  arXiv  Detail & Related papers  (2026-06-01T16:12:18Z)
• Internalizing Temporal Consistency in Video Object-Centric Learning without Explicit Regularization [34.98521123729049]
  Slot-Slot Contrastive (SSC) loss has become the cornerstone for state-of-the-art (SOTA) video Object-Centric Learning (OCL) methods.<n>We propose a paradigm shift: temporal consistency is better enforced as an implicit model design rather than an explicit loss.
  arXiv  Detail & Related papers  (2026-05-29T16:28:21Z)
• Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory [79.01059178883817]
  IAMFlow is a training-free identity-aware memory framework that explicitly models and tracks persistent entity identities.<n>A VLM asynchronously verifies and refines attributes from rendered frames, enabling explicit entity tracking in place of implicit similarity-based matching.<n>NarraStream-Bench is a benchmark for narrative streaming video generation that features 324 multi-prompt scripts spanning six dimensions.
  arXiv  Detail & Related papers  (2026-05-18T17:54:34Z)
• AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation [55.94507360511886]
  We propose AnyCrowd, a Transformer-based video generation framework capable of scaling to an arbitrary number of characters.<n>Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement.<n>Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global-background coordination.
  arXiv  Detail & Related papers  (2026-03-16T15:25:04Z)
• ObjectAlign: Neuro-Symbolic Object Consistency Verification and Correction [25.651102245332766]
  We introduce ObjectAlign, a framework that blends perceptual metrics with symbolic reasoning to detect, verify, and correct inconsistencies in edited video sequences.<n>We show up to 1.4 point improvement in CLIP Score and up to 6.1 point improvement in warp error compared to SOTA baselines on the DAVIS and Pexels video datasets.
  arXiv  Detail & Related papers  (2025-11-24T02:50:01Z)
• Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective [16.541717037293278]
  We introduce LIBERO-Mem, a non-Markovian task suite for stress-testing robotic manipulation under object-level partial observability.<n>It combines short- and long-horizon object tracking with temporally sequenced subgoals, requiring reasoning beyond the current frame.<n>We propose Embodied-SlotSSM, a slot-centric VLA framework built for temporal scalability.
  arXiv  Detail & Related papers  (2025-11-14T16:56:01Z)
• Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations [131.33758144860988]
  Identity-preserving text-to-video (IPT2V) generation aims to create high-fidelity videos with consistent human identity.<n>Current end-to-end frameworks suffer a critical spatial-temporal trade-off.<n>We propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics.
  arXiv  Detail & Related papers  (2025-07-07T06:54:44Z)
• Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases [69.46487306858789]
  Conditional Autoregressive Slot Attention (CA-SA) is a framework that enhances the temporal consistency of extracted object-centric representations in video-centric vision tasks. We present qualitative and quantitative results showing that our proposed method outperforms the considered baselines on downstream tasks.
  arXiv  Detail & Related papers  (2024-10-21T07:44:44Z)
• Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation [76.68301884987348]
  We propose a simple yet effective approach for self-supervised video object segmentation (VOS) Our key insight is that the inherent structural dependencies present in DINO-pretrained Transformers can be leveraged to establish robust-temporal segmentation correspondences in videos. Our method demonstrates state-of-the-art performance across multiple unsupervised VOS benchmarks and excels in complex real-world multi-object video segmentation tasks.
  arXiv  Detail & Related papers  (2023-11-29T18:47:17Z)
• Divided Attention: Unsupervised Multi-Object Discovery with Contextually Separated Slots [65.302728042116]
  We investigate the emergence of objects in visual perception in the absence of any semantic annotation.<n>The resulting model has received no supervision, does not use any pre-trained features, and yet it can segment the domain of an image into multiple moving regions.<n>The resulting motion segmentation method can handle an unknown and varying number of objects in real-time.
  arXiv  Detail & Related papers  (2023-04-04T00:26:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.