Grounded Object Centric Learning

• URL: http://arxiv.org/abs/2307.09437v2
• Date: Thu, 25 Jan 2024 15:52:19 GMT
• Title: Grounded Object Centric Learning
• Authors: Avinash Kori, Francesco Locatello, Fabio De Sousa Ribeiro, Francesca Toni, Ben Glocker
• Abstract summary: We present emphtextscConditional textscSlot textscAttention (textscCoSA) using a novel concept of emphGrounded Slot Dictionary (GSD) inspired by vector quantization. We demonstrate the benefits of our method in multiple downstream tasks such as scene generation, composition, and task adaptation.
• Score: 46.091323528165205
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The extraction of modular object-centric representations for downstream tasks is an emerging area of research. Learning grounded representations of objects that are guaranteed to be stable and invariant promises robust performance across different tasks and environments. Slot Attention (SA) learns object-centric representations by assigning objects to \textit{slots}, but presupposes a \textit{single} distribution from which all slots are randomly initialised. This results in an inability to learn \textit{specialized} slots which bind to specific object types and remain invariant to identity-preserving changes in object appearance. To address this, we present \emph{\textsc{Co}nditional \textsc{S}lot \textsc{A}ttention} (\textsc{CoSA}) using a novel concept of \emph{Grounded Slot Dictionary} (GSD) inspired by vector quantization. Our proposed GSD comprises (i) canonical object-level property vectors and (ii) parametric Gaussian distributions, which define a prior over the slots. We demonstrate the benefits of our method in multiple downstream tasks such as scene generation, composition, and task adaptation, whilst remaining competitive with SA in popular object discovery benchmarks.

Related papers

• ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting [54.92763171355442]
  ObjectGS is an object-aware framework that unifies 3D scene reconstruction with semantic understanding.<n>We show through experiments that ObjectGS outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks.
  arXiv  Detail & Related papers  (2025-07-21T10:06:23Z)
• Learning Spatial-Semantic Features for Robust Video Object Segmentation [108.045326229865]
  We propose a robust video object segmentation framework equipped with spatial-semantic features and discriminative object queries. We show that the proposed method set a new state-of-the-art performance on multiple datasets.
  arXiv  Detail & Related papers  (2024-07-10T15:36:00Z)
• Neural Slot Interpreters: Grounding Object Semantics in Emergent Slot Representations [4.807052027638089]
  We present the Neural Slot Interpreter (NSI) that learns to ground and generate object semantics via slot representations. NSI is an XML-like programming language that uses simple syntax rules to organize the object semantics of a scene into object-centric program primitives.
  arXiv  Detail & Related papers  (2024-02-02T12:37:23Z)
• Object-Centric Multiple Object Tracking [124.30650395969126]
  This paper proposes a video object-centric model for multiple-object tracking pipelines. It consists of an index-merge module that adapts the object-centric slots into detection outputs and an object memory module. Benefited from object-centric learning, we only require sparse detection labels for object localization and feature binding.
  arXiv  Detail & Related papers  (2023-09-01T03:34:12Z)
• Enhancing Interpretable Object Abstraction via Clustering-based Slot Initialization [17.25953277219166]
  We present a new method for object-centric representations using slots. Our method outperforms prior works consistently. We evaluate our method on object discovery and novel view synthesis tasks with various datasets.
  arXiv  Detail & Related papers  (2023-08-22T11:48:43Z)
• Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos [63.94040814459116]
  Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. We propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations.
  arXiv  Detail & Related papers  (2023-08-19T09:12:13Z)
• An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning [0.46644955105516456]
  Zero-shot Learning (ZSL) classification categorizes or predicts classes (labels) that are not included in the training set (unseen classes) Recent works proposed different semantic autoencoder (SAE) models where the encoder embeds a visual feature space into the semantic space and the decoder reconstructs the original visual feature space. We propose an integral projection-based semantic autoencoder (IP-SAE) where an encoder projects a visual feature space vectord with the semantic space into a latent representation space.
  arXiv  Detail & Related papers  (2023-06-26T12:06:20Z)
• Interpreting Embedding Spaces by Conceptualization [2.620130580437745]
  We present a novel method of understanding embeddings by transforming a latent embedding space into a comprehensible conceptual space. We devise a new evaluation method, using either human rater or LLM-based raters, to show that the vectors indeed represent the semantics of the original latent ones.
  arXiv  Detail & Related papers  (2022-08-22T15:32:17Z)
• Fine-Grained Object Classification via Self-Supervised Pose Alignment [42.55938966190932]
  We learn a novel graph based object representation to reveal a global configuration of local parts for self-supervised pose alignment across classes. We evaluate our method on three popular fine-grained object classification benchmarks, consistently achieving the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T01:46:19Z)
• Generalized Funnelling: Ensemble Learning and Heterogeneous Document Embeddings for Cross-Lingual Text Classification [78.83284164605473]
  emphFunnelling (Fun) is a recently proposed method for cross-lingual text classification. We describe emphGeneralized Funnelling (gFun) as a generalization of Fun. We show that gFun substantially improves over Fun and over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2021-09-17T23:33:04Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)
• Anchor & Transform: Learning Sparse Embeddings for Large Vocabularies [60.285091454321055]
  We design a simple and efficient embedding algorithm that learns a small set of anchor embeddings and a sparse transformation matrix. On text classification, language modeling, and movie recommendation benchmarks, we show that ANT is particularly suitable for large vocabulary sizes.
  arXiv  Detail & Related papers  (2020-03-18T13:07:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.