Contrastive Unsupervised Learning of World Model with Invariant Causal
Features
- URL: http://arxiv.org/abs/2209.14932v1
- Date: Thu, 29 Sep 2022 16:49:24 GMT
- Title: Contrastive Unsupervised Learning of World Model with Invariant Causal
Features
- Authors: Rudra P.K. Poudel, Harit Pandya, Roberto Cipolla
- Abstract summary: We present a world model, which learns causal features using the invariance principle.
We use contrastive unsupervised learning to learn the invariant causal features.
Our proposed model performs on par with the state-of-the-art counterpart.
- Score: 20.116319631571095
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper we present a world model, which learns causal features using
the invariance principle. In particular, we use contrastive unsupervised
learning to learn the invariant causal features, which enforces invariance
across augmentations of irrelevant parts or styles of the observation. The
world-model-based reinforcement learning methods independently optimize
representation learning and the policy. Thus naive contrastive loss
implementation collapses due to a lack of supervisory signals to the
representation learning module. We propose an intervention invariant auxiliary
task to mitigate this issue. Specifically, we utilize depth prediction to
explicitly enforce the invariance and use data augmentation as style
intervention on the RGB observation space. Our design leverages unsupervised
representation learning to learn the world model with invariant causal
features. Our proposed method significantly outperforms current
state-of-the-art model-based and model-free reinforcement learning methods on
out-of-distribution point navigation tasks on the iGibson dataset. Moreover,
our proposed model excels at the sim-to-real transfer of our perception
learning module. Finally, we evaluate our approach on the DeepMind control
suite and enforce invariance only implicitly since depth is not available.
Nevertheless, our proposed model performs on par with the state-of-the-art
counterpart.
Related papers
- Learning Non-Linear Invariants for Unsupervised Out-of-Distribution Detection [5.019613806273252]
We propose a framework consisting of a normalizing flow-like architecture capable of learning non-linear invariants.
Our approach achieves state-of-the-art results on an extensive U-OOD benchmark.
arXiv Detail & Related papers (2024-07-04T16:01:21Z) - A Probabilistic Model behind Self-Supervised Learning [53.64989127914936]
In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels.
We present a generative latent variable model for self-supervised learning.
We show that several families of discriminative SSL induce a comparable distribution over representations.
arXiv Detail & Related papers (2024-02-02T13:31:17Z) - ReCoRe: Regularized Contrastive Representation Learning of World Model [21.29132219042405]
We present a world model that learns invariant features using contrastive unsupervised learning and an intervention-invariant regularizer.
Our method outperforms current state-of-the-art model-based and model-free RL methods and significantly improves on out-of-distribution point navigation tasks evaluated on the iGibson benchmark.
arXiv Detail & Related papers (2023-12-14T15:53:07Z) - ProtoVAE: Prototypical Networks for Unsupervised Disentanglement [1.6114012813668934]
We introduce a novel deep generative VAE-based model, ProtoVAE, that leverages a deep metric learning Prototypical network trained using self-supervision.
Our model is completely unsupervised and requires no priori knowledge of the dataset, including the number of factors.
We evaluate our proposed model on the benchmark dSprites, 3DShapes, and MPI3D disentanglement datasets.
arXiv Detail & Related papers (2023-05-16T01:29:26Z) - Modeling Uncertain Feature Representation for Domain Generalization [49.129544670700525]
We show that our method consistently improves the network generalization ability on multiple vision tasks.
Our methods are simple yet effective and can be readily integrated into networks without additional trainable parameters or loss constraints.
arXiv Detail & Related papers (2023-01-16T14:25:02Z) - Why Do Self-Supervised Models Transfer? Investigating the Impact of
Invariance on Downstream Tasks [79.13089902898848]
Self-supervised learning is a powerful paradigm for representation learning on unlabelled images.
We show that different tasks in computer vision require features to encode different (in)variances.
arXiv Detail & Related papers (2021-11-22T18:16:35Z) - Influence Tuning: Demoting Spurious Correlations via Instance
Attribution and Instance-Driven Updates [26.527311287924995]
influence tuning can help deconfounding the model from spurious patterns in data.
We show that in a controlled setup, influence tuning can help deconfounding the model from spurious patterns in data.
arXiv Detail & Related papers (2021-10-07T06:59:46Z) - Model-Invariant State Abstractions for Model-Based Reinforcement
Learning [54.616645151708994]
We introduce a new type of state abstraction called textitmodel-invariance.
This allows for generalization to novel combinations of unseen values of state variables.
We prove that an optimal policy can be learned over this model-invariance state abstraction.
arXiv Detail & Related papers (2021-02-19T10:37:54Z) - Joint Generative and Contrastive Learning for Unsupervised Person
Re-identification [15.486689594217273]
Recent self-supervised contrastive learning provides an effective approach for unsupervised person re-identification (ReID)
In this paper, we incorporate a Generative Adversarial Network (GAN) and a contrastive learning module into one joint training framework.
arXiv Detail & Related papers (2020-12-16T16:49:57Z) - On the Benefits of Invariance in Neural Networks [56.362579457990094]
We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation.
We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
arXiv Detail & Related papers (2020-05-01T02:08:58Z) - Plannable Approximations to MDP Homomorphisms: Equivariance under
Actions [72.30921397899684]
We introduce a contrastive loss function that enforces action equivariance on the learned representations.
We prove that when our loss is zero, we have a homomorphism of a deterministic Markov Decision Process.
We show experimentally that for deterministic MDPs, the optimal policy in the abstract MDP can be successfully lifted to the original MDP.
arXiv Detail & Related papers (2020-02-27T08:29:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.