Finding Distributed Object-Centric Properties in Self-Supervised Transformers

• URL: http://arxiv.org/abs/2603.26127v1
• Date: Fri, 27 Mar 2026 07:22:04 GMT
• Title: Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• Abstract summary: Self-supervised Vision Transformers (ViTs) show an emergent ability to discover objects, typically observed in [] token attention maps of the final layer.<n>This is because the [] token, trained on an image-level objective, summarizes the entire image instead of focusing on objects.<n>We present Object-DINO, a training-free method that extracts this distributed object-centric information.
• Score: 59.00547715011873
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised Vision Transformers (ViTs) like DINO show an emergent ability to discover objects, typically observed in [CLS] token attention maps of the final layer. However, these maps often contain spurious activations resulting in poor localization of objects. This is because the [CLS] token, trained on an image-level objective, summarizes the entire image instead of focusing on objects. This aggregation dilutes the object-centric information existing in the local, patch-level interactions. We analyze this by computing inter-patch similarity using patch-level attention components (query, key, and value) across all layers. We find that: (1) Object-centric properties are encoded in the similarity maps derived from all three components ($q, k, v$), unlike prior work that uses only key features or the [CLS] token. (2) This object-centric information is distributed across the network, not just confined to the final layer. Based on these insights, we introduce Object-DINO, a training-free method that extracts this distributed object-centric information. Object-DINO clusters attention heads across all layers based on the similarities of their patches and automatically identifies the object-centric cluster corresponding to all objects. We demonstrate Object-DINO's effectiveness on two applications: enhancing unsupervised object discovery (+3.6 to +12.4 CorLoc gains) and mitigating object hallucination in Multimodal Large Language Models by providing visual grounding. Our results demonstrate that using this distributed object-centric information improves downstream tasks without additional training.

Related papers

• Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers? [11.078284195461578]
  We show that object binding emerges in pre-trained Vision Transformers (ViTs)<n>We find that IsSameObject is encoded in a low-dimensional subspace on top of object features, and that this signal actively guides attention.<n>Our findings challenge the view that ViTs lack object binding and highlight how symbolic knowledge of "which parts belong together" emerges naturally in a connectionist system.
  arXiv  Detail & Related papers  (2025-10-28T17:57:05Z)
• Object-level Correlation for Few-Shot Segmentation [37.247888341920984]
  Few-shot semantic segmentation (FSS) aims to segment objects of novel categories in the query images given only a few annotated support samples.<n>We propose an Object-level Correlation Network (OCNet) by establishing the object-level correlation between the support target object and query general objects.<n>The generated object-level correlation can mine the query target feature and suppress the hard pixel noise for the final prediction.
  arXiv  Detail & Related papers  (2025-09-09T16:58:28Z)
• Are We Done with Object-Centric Learning? [65.67948794110212]
  Object-centric learning (OCL) seeks to learn representations that only encode an object, isolated from other objects or background cues in a scene.<n>With recent sample-efficient segmentation models, we can separate objects in the pixel space and encode them independently.<n>We address the OOD generalization challenge caused by spurious background cues through the lens of OCL.
  arXiv  Detail & Related papers  (2025-04-09T17:59:05Z)
• Object-Centric Multiple Object Tracking [124.30650395969126]
  This paper proposes a video object-centric model for multiple-object tracking pipelines. It consists of an index-merge module that adapts the object-centric slots into detection outputs and an object memory module. Benefited from object-centric learning, we only require sparse detection labels for object localization and feature binding.
  arXiv  Detail & Related papers  (2023-09-01T03:34:12Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Complex-Valued Autoencoders for Object Discovery [62.26260974933819]
  We propose a distributed approach to object-centric representations: the Complex AutoEncoder. We show that this simple and efficient approach achieves better reconstruction performance than an equivalent real-valued autoencoder on simple multi-object datasets. We also show that it achieves competitive unsupervised object discovery performance to a SlotAttention model on two datasets, and manages to disentangle objects in a third dataset where SlotAttention fails - all while being 7-70 times faster to train.
  arXiv  Detail & Related papers  (2022-04-05T09:25:28Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.