Improved Representation of Asymmetrical Distances with Interval
Quasimetric Embeddings
- URL: http://arxiv.org/abs/2211.15120v2
- Date: Fri, 5 Jan 2024 19:27:59 GMT
- Title: Improved Representation of Asymmetrical Distances with Interval
Quasimetric Embeddings
- Authors: Tongzhou Wang, Phillip Isola
- Abstract summary: Asymmetrical distance structures (quasimetrics) are ubiquitous in our lives and are gaining more attention in machine learning applications.
We present four desirable properties in such quasimetric models, and show how prior works fail at them.
We propose Interval Quasimetric Embedding (IQE), which is designed to satisfy all four criteria.
- Score: 45.69333765438636
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Asymmetrical distance structures (quasimetrics) are ubiquitous in our lives
and are gaining more attention in machine learning applications. Imposing such
quasimetric structures in model representations has been shown to improve many
tasks, including reinforcement learning (RL) and causal relation learning. In
this work, we present four desirable properties in such quasimetric models, and
show how prior works fail at them. We propose Interval Quasimetric Embedding
(IQE), which is designed to satisfy all four criteria. On three quasimetric
learning experiments, IQEs show strong approximation and generalization
abilities, leading to better performance and improved efficiency over prior
methods.
Project Page: https://www.tongzhouwang.info/interval_quasimetric_embedding
Quasimetric Learning Code Package:
https://www.github.com/quasimetric-learning/torch-quasimetric
Related papers
- Symmetry Considerations for Learning Task Symmetric Robot Policies [12.856889419651521]
Symmetry is a fundamental aspect of many real-world robotic tasks.
Current deep reinforcement learning (DRL) approaches can seldom harness and exploit symmetry effectively.
arXiv Detail & Related papers (2024-03-07T09:41:11Z) - A Generative Model of Symmetry Transformations [44.87295754993983]
We build a generative model that explicitly aims to capture the data's approximate symmetries.
We empirically demonstrate its ability to capture symmetries under affine and color transformations.
arXiv Detail & Related papers (2024-03-04T11:32:18Z) - Addressing Imperfect Symmetry: a Novel Symmetry-Learning Actor-Critic
Extension [0.46040036610482665]
We introduce Adaptive Symmetry (ASL) $x2013$ a model-minimization actor-critic extension that addresses incomplete symmetry.
ASL consists of symmetry fitting component and modular loss function that enforces a common relation across all states while adapting to the learned policy.
The results demonstrate that ASL is capable of recovering from large perturbations and generalizing to hidden symmetric states.
arXiv Detail & Related papers (2023-09-06T04:47:46Z) - ${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning [7.712824077083934]
We focus on exploiting Euclidean symmetries inherent in certain cooperative multi-agent reinforcement learning problems.
We design neural network architectures with symmetric constraints embedded as an inductive bias for multi-agent actor-critic methods.
arXiv Detail & Related papers (2023-08-23T00:18:17Z) - Understanding Augmentation-based Self-Supervised Representation Learning
via RKHS Approximation and Regression [53.15502562048627]
Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator.
This work delves into a statistical analysis of augmentation-based pretraining.
arXiv Detail & Related papers (2023-06-01T15:18:55Z) - Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning [73.80728148866906]
Quasimetric Reinforcement Learning (QRL) is a new RL method that utilizes quasimetric models to learn optimal value functions.
On offline and online goal-reaching benchmarks, QRL also demonstrates improved sample efficiency and performance.
arXiv Detail & Related papers (2023-04-03T17:59:58Z) - SPE: Symmetrical Prompt Enhancement for Fact Probing [81.82104239636574]
We propose a continuous prompt-based method for factual probing in pretrained language models (PLMs)
Our results on a popular factual probing dataset, LAMA, show significant improvement of SPE over previous probing methods.
arXiv Detail & Related papers (2022-11-14T03:05:41Z) - On the Learning and Learnablity of Quasimetrics [32.0469500831667]
In reinforcement learning and control, optimal goal-reaching strategies are rarely reversible (symmetrical)
Despite their common appearance, little research has been done on the learning of quasimetrics.
Our proposed Poisson Quasimetric Embedding (PQE) is the first quasimetric learning formulation that both is learnable with gradient-based optimization.
arXiv Detail & Related papers (2022-06-30T17:59:52Z) - Hyperbolic Vision Transformers: Combining Improvements in Metric
Learning [116.13290702262248]
We propose a new hyperbolic-based model for metric learning.
At the core of our method is a vision transformer with output embeddings mapped to hyperbolic space.
We evaluate the proposed model with six different formulations on four datasets.
arXiv Detail & Related papers (2022-03-21T09:48:23Z) - Online Target Q-learning with Reverse Experience Replay: Efficiently
finding the Optimal Policy for Linear MDPs [50.75812033462294]
We bridge the gap between practical success of Q-learning and pessimistic theoretical results.
We present novel methods Q-Rex and Q-RexDaRe.
We show that Q-Rex efficiently finds the optimal policy for linear MDPs.
arXiv Detail & Related papers (2021-10-16T01:47:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.