On Equivariant and Invariant Learning of Object Landmark Representations
- URL: http://arxiv.org/abs/2006.14787v2
- Date: Fri, 2 Apr 2021 20:23:49 GMT
- Title: On Equivariant and Invariant Learning of Object Landmark Representations
- Authors: Zezhou Cheng, Jong-Chyi Su, Subhransu Maji
- Abstract summary: We develop a simple and effective approach by combining instance-discriminative and spatially-discriminative contrastive learning.
We show that when a deep network is trained to be invariant to geometric and photometric transformations, representations emerge from its intermediate layers that are highly predictive of object landmarks.
- Score: 36.214069685880986
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Given a collection of images, humans are able to discover landmarks by
modeling the shared geometric structure across instances. This idea of
geometric equivariance has been widely used for the unsupervised discovery of
object landmark representations. In this paper, we develop a simple and
effective approach by combining instance-discriminative and
spatially-discriminative contrastive learning. We show that when a deep network
is trained to be invariant to geometric and photometric transformations,
representations emerge from its intermediate layers that are highly predictive
of object landmarks. Stacking these across layers in a "hypercolumn" and
projecting them using spatially-contrastive learning further improves their
performance on matching and few-shot landmark regression tasks. We also present
a unified view of existing equivariant and invariant representation learning
approaches through the lens of contrastive learning, shedding light on the
nature of invariances learned. Experiments on standard benchmarks for landmark
learning, as well as a new challenging one we propose, show that the proposed
approach surpasses prior state-of-the-art.
Related papers
- Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning [5.802794302956837]
Euclidean deep learning is often inadequate for addressing real-world signals where the representation space is irregular and curved with complex topologies.
We focus on the importance of symmetry group equivariant deep learning models and their realization of convolution-like operations on graphs, 3D shapes, and non-Euclidean spaces.
arXiv Detail & Related papers (2024-09-11T15:07:18Z) - Object-centric architectures enable efficient causal representation
learning [51.6196391784561]
We show that when the observations are of multiple objects, the generative function is no longer injective and disentanglement fails in practice.
We develop an object-centric architecture that leverages weak supervision from sparse perturbations to disentangle each object's properties.
This approach is more data-efficient in the sense that it requires significantly fewer perturbations than a comparable approach that encodes to a Euclidean space.
arXiv Detail & Related papers (2023-10-29T16:01:03Z) - LEAD: Self-Supervised Landmark Estimation by Aligning Distributions of
Feature Similarity [49.84167231111667]
Existing works in self-supervised landmark detection are based on learning dense (pixel-level) feature representations from an image.
We introduce an approach to enhance the learning of dense equivariant representations in a self-supervised fashion.
We show that having such a prior in the feature extractor helps in landmark detection, even under drastically limited number of annotations.
arXiv Detail & Related papers (2022-04-06T17:48:18Z) - Leveraging Equivariant Features for Absolute Pose Regression [9.30597356471664]
We show that a translation and rotation equivariant Convolutional Neural Network directly induces representations of camera motions into the feature space.
We then show that this geometric property allows for implicitly augmenting the training data under a whole group of image plane-preserving transformations.
arXiv Detail & Related papers (2022-04-05T12:44:20Z) - Fusing Local Similarities for Retrieval-based 3D Orientation Estimation
of Unseen Objects [70.49392581592089]
We tackle the task of estimating the 3D orientation of previously-unseen objects from monocular images.
We follow a retrieval-based strategy and prevent the network from learning object-specific features.
Our experiments on the LineMOD, LineMOD-Occluded, and T-LESS datasets show that our method yields a significantly better generalization to unseen objects than previous works.
arXiv Detail & Related papers (2022-03-16T08:53:00Z) - Digging Into Self-Supervised Learning of Feature Descriptors [14.47046413243358]
We propose a set of improvements that combined lead to powerful feature descriptors.
We show that increasing the search space from in-pair to in-batch for hard negative mining brings consistent improvement.
We demonstrate that a combination of synthetic homography transformation, color augmentation, and photorealistic image stylization produces useful representations.
arXiv Detail & Related papers (2021-10-10T12:22:44Z) - Geometry-Aware Self-Training for Unsupervised Domain Adaptationon Object
Point Clouds [36.49322708074682]
This paper proposes a new method of geometry-aware self-training (GAST) for unsupervised domain adaptation of object point cloud classification.
Specifically, this paper aims to learn a domain-shared representation of semantic categories, via two novel self-supervised geometric learning tasks as feature regularization.
On the other hand, a diverse point distribution across datasets can be normalized with a novel curvature-aware distortion localization.
arXiv Detail & Related papers (2021-08-20T13:29:11Z) - Instance Similarity Learning for Unsupervised Feature Representation [83.31011038813459]
We propose an instance similarity learning (ISL) method for unsupervised feature representation.
We employ the Generative Adversarial Networks (GAN) to mine the underlying feature manifold.
Experiments on image classification demonstrate the superiority of our method compared with the state-of-the-art methods.
arXiv Detail & Related papers (2021-08-05T16:42:06Z) - Pretrained equivariant features improve unsupervised landmark discovery [69.02115180674885]
We formulate a two-step unsupervised approach that overcomes this challenge by first learning powerful pixel-based features.
Our method produces state-of-the-art results in several challenging landmark detection datasets.
arXiv Detail & Related papers (2021-04-07T05:42:11Z) - Quadric hypersurface intersection for manifold learning in feature space [52.83976795260532]
manifold learning technique suitable for moderately high dimension and large datasets.
The technique is learned from the training data in the form of an intersection of quadric hypersurfaces.
At test time, this manifold can be used to introduce an outlier score for arbitrary new points.
arXiv Detail & Related papers (2021-02-11T18:52:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.