Asymmetric Norms to Approximate the Minimum Action Distance

• URL: http://arxiv.org/abs/2312.10276v2
• Date: Tue, 19 Dec 2023 08:12:51 GMT
• Title: Asymmetric Norms to Approximate the Minimum Action Distance
• Authors: Lorenzo Steccanella, Anders Jonsson
• Abstract summary: This paper presents a state representation for reward-free Markov decision processes. We show how this representation can be leveraged to learn goal-conditioned policies.
• Score: 9.040428950629153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a state representation for reward-free Markov decision processes. The idea is to learn, in a self-supervised manner, an embedding space where distances between pairs of embedded states correspond to the minimum number of actions needed to transition between them. Unlike previous methods, our approach incorporates an asymmetric norm parametrization, enabling accurate approximations of minimum action distances in environments with inherent asymmetry. We show how this representation can be leveraged to learn goal-conditioned policies, providing a notion of similarity between states and goals and a useful heuristic distance to guide planning. To validate our approach, we conduct empirical experiments on both symmetric and asymmetric environments. Our results show that our asymmetric norm parametrization performs comparably to symmetric norms in symmetric environments and surpasses symmetric norms in asymmetric environments.

Related papers

• Symmetric Behavior Regularization via Taylor Expansion of Symmetry [8.032060509915821]
  We show that symmetric divergences do not permit an analytic policy as regularization and can incur numerical issues as loss.<n>We propose Symmetric $f$ Actor-Critic (S$f$-AC), the first practical BRPO algorithm with symmetric divergences.
  arXiv  Detail & Related papers  (2025-08-06T09:01:29Z)
• Topological nature of edge states for one-dimensional systems without symmetry protection [46.87902365052209]
  We numerically verify and analytically prove a winding number invariant that correctly predicts the number of edge states in one-dimensional, nearest-neighbour (between unit cells) Our invariant is invariant under unitary and similarity transforms.
  arXiv  Detail & Related papers  (2024-12-13T19:44:54Z)
• SymmetryLens: A new candidate paradigm for unsupervised symmetry learning via locality and equivariance [0.0]
  We develop a new, unsupervised symmetry learning method that starts with raw data. We demonstrate that this coupling between symmetry and locality, together with a special optimization technique developed for entropy estimation, results in a highly stable system. The symmetry actions we consider are group representations, however, we believe the approach has the potential to be generalized to more general, nonlinear actions of non-commutative Lie groups.
  arXiv  Detail & Related papers  (2024-10-07T17:40:51Z)
• Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimization [45.935798913942904]
  The effect of relative entropy asymmetry is analyzed in the context of empirical risk minimization. By comparing the well-understood Type-I ERM-RER with Type-II ERM-RER, the effects of entropy asymmetry are highlighted. It is shown that Type-II regularization is equivalent to Type-I regularization with an appropriate transformation of the empirical risk function.
  arXiv  Detail & Related papers  (2024-10-02T09:43:43Z)
• Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks [43.88179780450706]
  We investigate the impact of weight space permutation symmetries on variational inference. We devise a symmetric symmetrization mechanism for constructing permutation invariant variational posteriors. We show that the symmetrized distribution has a strictly better fit to the true posterior, and that it can be trained using the original ELBO objective.
  arXiv  Detail & Related papers  (2024-08-10T09:06:34Z)
• How much symmetry do symmetric measurements need for efficient operational applications? [0.0]
  For informationally complete sets, we propose construction methods from orthonormal Hermitian operator bases. Some of the symmetry properties, lost in the process of generalization, can be recovered without fixing the same number of elements for all POVMs.
  arXiv  Detail & Related papers  (2024-04-02T15:23:08Z)
• A Generative Model of Symmetry Transformations [44.87295754993983]
  We build a generative model that explicitly aims to capture the data's approximate symmetries. We empirically demonstrate its ability to capture symmetries under affine and color transformations.
  arXiv  Detail & Related papers  (2024-03-04T11:32:18Z)
• Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetries [11.54837584979607]
  Group equivariance can overly constrain models if the symmetries in the group differ from those observed in data. We propose a method able to detect the level of symmetry of each input without the need for labels. Our framework is general enough to accommodate different families of both continuous and discrete symmetry distributions.
  arXiv  Detail & Related papers  (2023-12-19T15:11:46Z)
• Regularizing Towards Soft Equivariance Under Mixed Symmetries [23.603875905608565]
  We present a regularizer-based method for building a model for a dataset with mixed approximate symmetries. We show that our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.
  arXiv  Detail & Related papers  (2023-06-01T05:33:41Z)
• Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddings [45.69333765438636]
  Asymmetrical distance structures (quasimetrics) are ubiquitous in our lives and are gaining more attention in machine learning applications. We present four desirable properties in such quasimetric models, and show how prior works fail at them. We propose Interval Quasimetric Embedding (IQE), which is designed to satisfy all four criteria.
  arXiv  Detail & Related papers  (2022-11-28T08:22:26Z)
• Optimal variance-reduced stochastic approximation in Banach spaces [114.8734960258221]
  We study the problem of estimating the fixed point of a contractive operator defined on a separable Banach space. We establish non-asymptotic bounds for both the operator defect and the estimation error.
  arXiv  Detail & Related papers  (2022-01-21T02:46:57Z)
• Symmetry Breaking in Symmetric Tensor Decomposition [44.181747424363245]
  We consider the nonsymmetry problem associated with computing the points rank decomposition of symmetric tensors. We show that critical points the loss function is detected by standard methods.
  arXiv  Detail & Related papers  (2021-03-10T18:11:22Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)
• The quantum marginal problem for symmetric states: applications to variational optimization, nonlocality and self-testing [0.0]
  We present a method to solve the quantum marginal problem for symmetric $d$-level systems. We illustrate the applicability of the method in central quantum information problems with several exemplary case studies.
  arXiv  Detail & Related papers  (2020-01-13T18:20:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.