Exploiting Data Hierarchy as a New Modality for Contrastive Learning

• URL: http://arxiv.org/abs/2401.03312v1
• Date: Sat, 6 Jan 2024 21:47:49 GMT
• Title: Exploiting Data Hierarchy as a New Modality for Contrastive Learning
• Authors: Arjun Bhalla, Daniel Levenson, Jan Bernhard, Anton Abilov
• Abstract summary: This work investigates how hierarchically structured data can help neural networks learn conceptual representations of cathedrals. The underlying WikiScenes dataset provides a spatially organized hierarchical structure of cathedral components. We propose a novel hierarchical contrastive training approach that leverages a triplet margin loss to represent the data's spatial hierarchy in the encoder's latent space.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work investigates how hierarchically structured data can help neural networks learn conceptual representations of cathedrals. The underlying WikiScenes dataset provides a spatially organized hierarchical structure of cathedral components. We propose a novel hierarchical contrastive training approach that leverages a triplet margin loss to represent the data's spatial hierarchy in the encoder's latent space. As such, the proposed approach investigates if the dataset structure provides valuable information for self-supervised learning. We apply t-SNE to visualize the resultant latent space and evaluate the proposed approach by comparing it with other dataset-specific contrastive learning methods using a common downstream classification task. The proposed method outperforms the comparable weakly-supervised and baseline methods. Our findings suggest that dataset structure is a valuable modality for weakly-supervised learning.

Related papers

• From Logits to Hierarchies: Hierarchical Clustering made Simple [16.132657141993548]
  We show that a lightweight procedure implemented on top of pre-trained non-hierarchical clustering models outperforms models designed specifically for hierarchical clustering. Our proposed approach is computationally efficient and applicable to any pre-trained clustering model that outputs logits, without requiring any fine-tuning.
  arXiv  Detail & Related papers  (2024-10-10T12:27:45Z)
• Self-supervised Learning of Dense Hierarchical Representations for Medical Image Segmentation [2.2265038612930663]
  This paper demonstrates a self-supervised framework for learning voxel-wise coarse-to-fine representations tailored for dense downstream tasks. We devise a training strategy that balances the contributions of features from multiple scales, ensuring that the learned representations capture both coarse and fine-grained details.
  arXiv  Detail & Related papers  (2024-01-12T09:47:17Z)
• Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
  In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure. We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance. We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
  arXiv  Detail & Related papers  (2023-05-24T11:05:12Z)
• Structural hierarchical learning for energy networks [1.2599533416395767]
  This work investigates custom neural network designs inspired by the topological structures of hierarchies. Results unveil that, in a data-limited setting, structural models with fewer connections perform overall best. Overall, this work expands and improves hierarchical learning methods thanks to a structurally-scaled learning mechanism extension.
  arXiv  Detail & Related papers  (2023-02-08T10:28:32Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Leveraging Ensembles and Self-Supervised Learning for Fully-Unsupervised Person Re-Identification and Text Authorship Attribution [77.85461690214551]
  Learning from fully-unlabeled data is challenging in Multimedia Forensics problems, such as Person Re-Identification and Text Authorship Attribution. Recent self-supervised learning methods have shown to be effective when dealing with fully-unlabeled data in cases where the underlying classes have significant semantic differences. We propose a strategy to tackle Person Re-Identification and Text Authorship Attribution by enabling learning from unlabeled data even when samples from different classes are not prominently diverse.
  arXiv  Detail & Related papers  (2022-02-07T13:08:11Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Learning Representations of Hierarchical Slates in Collaborative Filtering [2.8145809047875066]
  We are interested in building collaborative filtering models for recommendation systems where users interact with slates instead of individual items. The central idea of our approach is to learn low dimensional embeddings of these slates. We present a novel way to learn these embeddings by making use of the (unknown) statistics of the underlying distribution generating the hierarchical data.
  arXiv  Detail & Related papers  (2020-09-25T18:34:02Z)
• Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene [76.4183572058063]
  We present a richly-annotated 3D point cloud dataset for multiple outdoor scene understanding tasks. The dataset has been point-wisely annotated with both hierarchical and instance-based labels. We formulate a hierarchical learning problem for 3D point cloud segmentation and propose a measurement evaluating consistency across various hierarchies.
  arXiv  Detail & Related papers  (2020-08-11T19:10:32Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• Leveraging Class Hierarchies with Metric-Guided Prototype Learning [5.070542698701158]
  In many classification tasks, the set of target classes can be organized into a hierarchy. This structure induces a semantic distance between classes, and can be summarised under the form of a cost matrix. We propose to model the hierarchical class structure by integrating this metric in the supervision of a prototypical network.
  arXiv  Detail & Related papers  (2020-07-06T20:22:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.