Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions

• URL: http://arxiv.org/abs/2207.12067v3
• Date: Tue, 2 Jul 2024 15:46:13 GMT
• Title: Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions
• Authors: Hamza Keurti, Hsiao-Ru Pan, Michel Besserve, Benjamin F. Grewe, Bernhard Schölkopf,
• Abstract summary: We propose methods enabling an agent acting upon the world to learn internal representations of sensory information consistent with actions that modify it. In contrast to existing work, our approach does not require prior knowledge of the group and does not restrict the set of actions the agent can perform.
• Score: 51.71245032890532
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How can agents learn internal models that veridically represent interactions with the real world is a largely open question. As machine learning is moving towards representations containing not just observational but also interventional knowledge, we study this problem using tools from representation learning and group theory. We propose methods enabling an agent acting upon the world to learn internal representations of sensory information that are consistent with actions that modify it. We use an autoencoder equipped with a group representation acting on its latent space, trained using an equivariance-derived loss in order to enforce a suitable homomorphism property on the group representation. In contrast to existing work, our approach does not require prior knowledge of the group and does not restrict the set of actions the agent can perform. We motivate our method theoretically, and show empirically that it can learn a group representation of the actions, thereby capturing the structure of the set of transformations applied to the environment. We further show that this allows agents to predict the effect of sequences of future actions with improved accuracy.

Related papers

• A Pattern Language for Machine Learning Tasks [0.0]
  We view objective functions as constraints on the behaviour of learners. We develop a formal graphical language that allows us to separate the core tasks of a behaviour from its implementation details. As proof-of-concept, we design a novel task that enables converting classifiers into generative models we call "manipulators"
  arXiv  Detail & Related papers  (2024-07-02T16:50:27Z)
• A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
  In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels. We present a generative latent variable model for self-supervised learning. We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
  arXiv  Detail & Related papers  (2024-02-02T13:31:17Z)
• Perceptual Group Tokenizer: Building Perception with Iterative Grouping [14.760204235027627]
  We propose the Perceptual Group Tokenizer, a model that relies on grouping operations to extract visual features and perform self-supervised representation learning. We show that the proposed model can achieve competitive computation performance compared to state-of-the-art vision architectures.
  arXiv  Detail & Related papers  (2023-11-30T07:00:14Z)
• What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? [57.92924256181857]
  We find that visual representations designed for manipulation and control tasks do not necessarily generalize under subtle changes in lighting and scene texture. We find that emergent segmentation ability is a strong predictor of out-of-distribution generalization among ViT models.
  arXiv  Detail & Related papers  (2023-11-03T18:09:08Z)
• Robust and Controllable Object-Centric Learning through Energy-based Models [95.68748828339059]
  ours is a conceptually simple and general approach to learning object-centric representations through an energy-based model. We show that ours can be easily integrated into existing architectures and can effectively extract high-quality object-centric representations.
  arXiv  Detail & Related papers  (2022-10-11T15:11:15Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams [64.82800502603138]
  This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream. The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations. Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream.
  arXiv  Detail & Related papers  (2022-04-26T09:52:31Z)
• Observing a group to infer individual characteristics [1.0152838128195465]
  We propose a new observer algorithm that infers, based only on observed movement information, how the local neighborhood aids or hinders agent movement. Unlike a traditional supervised learning approach, this algorithm is based on physical insights and scaling arguments, and does not rely on training-data. Data-agnostic approaches like this have relevance to a large class of real-world problems where clean, labeled data is difficult to obtain.
  arXiv  Detail & Related papers  (2021-10-12T09:59:54Z)
• Domain-Robust Visual Imitation Learning with Mutual Information Constraints [0.0]
  We introduce a new algorithm called Disentangling Generative Adversarial Imitation Learning (DisentanGAIL) Our algorithm enables autonomous agents to learn directly from high dimensional observations of an expert performing a task.
  arXiv  Detail & Related papers  (2021-03-08T21:18:58Z)
• On the Sensory Commutativity of Action Sequences for Embodied Agents [2.320417845168326]
  We study perception for embodied agents under the mathematical formalism of group theory. We introduce the Sensory Commutativity Probability criterion which measures how much an agent's degree of freedom affects the environment. We empirically illustrate how SCP and the commutative properties of action sequences can be used to learn about objects in the environment.
  arXiv  Detail & Related papers  (2020-02-13T16:58:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.