Efficient Object-centric Representation Learning with Pre-trained Geometric Prior

• URL: http://arxiv.org/abs/2412.12331v1
• Date: Mon, 16 Dec 2024 20:01:35 GMT
• Title: Efficient Object-centric Representation Learning with Pre-trained Geometric Prior
• Authors: Phúc H. Le Khac, Graham Healy, Alan F. Smeaton,
• Abstract summary: We propose a weakly-supervised framework that emphasises geometric understanding and leverages pre-trained vision models to enhance object discovery. Our method introduces an efficient slot decoder specifically designed for object-centric learning, enabling effective representation of multi-object scenes without requiring explicit depth information.
• Score: 1.9685736810241874
• License:
• Abstract: This paper addresses key challenges in object-centric representation learning of video. While existing approaches struggle with complex scenes, we propose a novel weakly-supervised framework that emphasises geometric understanding and leverages pre-trained vision models to enhance object discovery. Our method introduces an efficient slot decoder specifically designed for object-centric learning, enabling effective representation of multi-object scenes without requiring explicit depth information. Results on synthetic video benchmarks with increasing complexity in terms of objects and their movement, object occlusion and camera motion demonstrate that our approach achieves comparable performance to supervised methods while maintaining computational efficiency. This advances the field towards more practical applications in complex real-world scenarios.

Related papers

• Zero-Shot Object-Centric Representation Learning [72.43369950684057]
  We study current object-centric methods through the lens of zero-shot generalization. We introduce a benchmark comprising eight different synthetic and real-world datasets. We find that training on diverse real-world images improves transferability to unseen scenarios.
  arXiv  Detail & Related papers  (2024-08-17T10:37:07Z)
• Rethinking Image-to-Video Adaptation: An Object-centric Perspective [61.833533295978484]
  We propose a novel and efficient image-to-video adaptation strategy from the object-centric perspective. Inspired by human perception, we integrate a proxy task of object discovery into image-to-video transfer learning.
  arXiv  Detail & Related papers  (2024-07-09T13:58:10Z)
• Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMs [53.66070434419739]
  Generalizable articulated object manipulation is essential for home-assistant robots. We propose a kinematic-aware prompting framework that prompts Large Language Models with kinematic knowledge of objects to generate low-level motion waypoints. Our framework outperforms traditional methods on 8 categories seen and shows a powerful zero-shot capability for 8 unseen articulated object categories.
  arXiv  Detail & Related papers  (2023-11-06T03:26:41Z)
• Graphical Object-Centric Actor-Critic [55.2480439325792]
  We propose a novel object-centric reinforcement learning algorithm combining actor-critic and model-based approaches. We use a transformer encoder to extract object representations and graph neural networks to approximate the dynamics of an environment. Our algorithm performs better in a visually complex 3D robotic environment and a 2D environment with compositional structure than the state-of-the-art model-free actor-critic algorithm.
  arXiv  Detail & Related papers  (2023-10-26T06:05:12Z)
• Cycle Consistency Driven Object Discovery [75.60399804639403]
  We introduce a method that explicitly optimize the constraint that each object in a scene should be associated with a distinct slot. By integrating these consistency objectives into various existing slot-based object-centric methods, we showcase substantial improvements in object-discovery performance. Our results suggest that the proposed approach not only improves object discovery, but also provides richer features for downstream tasks.
  arXiv  Detail & Related papers  (2023-06-03T21:49:06Z)
• Robust and Controllable Object-Centric Learning through Energy-based Models [95.68748828339059]
  ours is a conceptually simple and general approach to learning object-centric representations through an energy-based model. We show that ours can be easily integrated into existing architectures and can effectively extract high-quality object-centric representations.
  arXiv  Detail & Related papers  (2022-10-11T15:11:15Z)
• Object Pursuit: Building a Space of Objects via Discriminative Weight Generation [23.85039747700698]
  We propose a framework to continuously learn object-centric representations for visual learning and understanding. We leverage interactions to sample diverse variations of an object and the corresponding training signals while learning the object-centric representations. We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.
  arXiv  Detail & Related papers  (2021-12-15T08:25:30Z)
• Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning [0.0951828574518325]
  Current deep reinforcement learning (RL) approaches incorporate minimal prior knowledge about the environment. We propose a framework for reasoning about object dynamics and behavior to rapidly determine minimal and task-specific object representations. We also highlight the potential of this framework on several Atari games, using our object representation and standard RL and planning algorithms to learn dramatically faster than existing deep RL algorithms.
  arXiv  Detail & Related papers  (2020-03-03T08:18:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.