Triangular Contrastive Learning on Molecular Graphs

• URL: http://arxiv.org/abs/2205.13279v1
• Date: Thu, 26 May 2022 11:34:08 GMT
• Title: Triangular Contrastive Learning on Molecular Graphs
• Authors: MinGyu Choi, Wonseok Shin, Yijingxiu Lu, Sun Kim
• Abstract summary: Triangular Contrastive Learning (TriCL) is a universal framework for trimodal contrastive learning. Triangular Area Loss is a novel intermodal contrastive loss that learns the angular geometry of the embedding space. We show that Triangular Area Loss can address the line-collapsing problem by discriminating modalities by angle.
• Score: 2.8331075191137463
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent contrastive learning methods have shown to be effective in various tasks, learning generalizable representations invariant to data augmentation thereby leading to state of the art performances. Regarding the multifaceted nature of large unlabeled data used in self-supervised learning while majority of real-word downstream tasks use single format of data, a multimodal framework that can train single modality to learn diverse perspectives from other modalities is an important challenge. In this paper, we propose TriCL (Triangular Contrastive Learning), a universal framework for trimodal contrastive learning. TriCL takes advantage of Triangular Area Loss, a novel intermodal contrastive loss that learns the angular geometry of the embedding space through simultaneously contrasting the area of positive and negative triplets. Systematic observation on embedding space in terms of alignment and uniformity showed that Triangular Area Loss can address the line-collapsing problem by discriminating modalities by angle. Our experimental results also demonstrate the outperformance of TriCL on downstream task of molecular property prediction which implies that the advantages of the embedding space indeed benefits the performance on downstream tasks.

Related papers

• Semi-Supervised Contrastive Learning with Orthonormal Prototypes [1.478364697333309]
  dimensional collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge.<n>We propose CLOP, a novel semi-supervised loss function designed to prevent dimensional collapse by promoting the formation of linear subspaces among class embeddings.<n>We show that CLOP improves performance in image classification and object detection tasks while also exhibiting greater stability across different learning rates and batch sizes.
  arXiv  Detail & Related papers  (2025-11-27T13:26:59Z)
• A Primer on SO(3) Action Representations in Deep Reinforcement Learning [6.964881957695288]
  We show that representation-induced geometry strongly influences exploration and optimization.<n>Our results highlight that representing actions as tangent vectors in the local frame yields the most reliable results across algorithms.
  arXiv  Detail & Related papers  (2025-10-13T07:49:21Z)
• MultiCo3D: Multi-Label Voxel Contrast for One-Shot Incremental Segmentation of 3D Neuroimages [12.060313392970695]
  3D neuroimages provide a comprehensive view of brain structure and function, aiding in precise localization and functional connectivity analysis. One-shot Class Incremental Semantic (OCIS) refers to segmenting new classes using only a single sample. We propose a new multi-label voxel contrast framework called MultiCo3D for one-shot class incremental tract segmentation.
  arXiv  Detail & Related papers  (2025-03-09T13:06:20Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning [52.70210390424605]
  In this work, we consider endowing a neural network autoencoder with three select inductive biases from the literature. In practice, however, naively combining existing techniques instantiating these inductive biases fails to yield significant benefits. We propose adaptations to the three techniques that simplify the learning problem, equip key regularization terms with stabilizing invariances, and quash degenerate incentives. The resulting model, Tripod, achieves state-of-the-art results on a suite of four image disentanglement benchmarks.
  arXiv  Detail & Related papers  (2024-04-16T04:52:41Z)
• The Common Stability Mechanism behind most Self-Supervised Learning Approaches [64.40701218561921]
  We provide a framework to explain the stability mechanism of different self-supervised learning techniques. We discuss the working mechanism of contrastive techniques like SimCLR, non-contrastive techniques like BYOL, SWAV, SimSiam, Barlow Twins, and DINO. We formulate different hypotheses and test them using the Imagenet100 dataset.
  arXiv  Detail & Related papers  (2024-02-22T20:36:24Z)
• Triple Disentangled Representation Learning for Multimodal Affective Analysis [20.37986194570143]
  Multimodal learning has exhibited a significant advantage in affective analysis tasks. Many emerging studies focus on disentangling the modality-invariant and modality-specific representations from input data and then fusing them for prediction. We propose a novel triple disentanglement approach, TriDiRA, which disentangles the modality-invariant, effective modality-specific and ineffective modality-specific representations from input data.
  arXiv  Detail & Related papers  (2024-01-29T12:45:27Z)
• Semantic Positive Pairs for Enhancing Visual Representation Learning of Instance Discrimination methods [4.680881326162484]
  Self-supervised learning algorithms (SSL) based on instance discrimination have shown promising results. We propose an approach to identify those images with similar semantic content and treat them as positive instances. We run experiments on three benchmark datasets: ImageNet, STL-10 and CIFAR-10 with different instance discrimination SSL approaches.
  arXiv  Detail & Related papers  (2023-06-28T11:47:08Z)
• Continual Vision-Language Representation Learning with Off-Diagonal Information [112.39419069447902]
  Multi-modal contrastive learning frameworks like CLIP typically require a large amount of image-text samples for training. This paper discusses the feasibility of continual CLIP training using streaming data.
  arXiv  Detail & Related papers  (2023-05-11T08:04:46Z)
• Instance-Variant Loss with Gaussian RBF Kernel for 3D Cross-modal Retriveal [52.41252219453429]
  Existing methods treat all instances equally, applying the same penalty strength to instances with varying degrees of difficulty. This can result in ambiguous convergence or local optima, severely compromising the separability of the feature space. We propose an Instance-Variant loss to assign different penalty strengths to different instances, improving the space separability.
  arXiv  Detail & Related papers  (2023-05-07T10:12:14Z)
• Cross-Stream Contrastive Learning for Self-Supervised Skeleton-Based Action Recognition [22.067143671631303]
  Self-supervised skeleton-based action recognition enjoys a rapid growth along with the development of contrastive learning. We propose a Cross-Stream Contrastive Learning framework for skeleton-based action Representation learning (CSCLR) Specifically, the proposed CSCLR not only utilizes intra-stream contrast pairs, but introduces inter-stream contrast pairs as hard samples to formulate a better representation learning.
  arXiv  Detail & Related papers  (2023-05-03T10:31:35Z)
• Multi-view Feature Extraction based on Triple Contrastive Heads [1.4467794332678539]
  We propose a novel multi-view feature extraction method based on triple contrastive heads. The proposed method offers a strong advantage for multi-view feature extraction.
  arXiv  Detail & Related papers  (2023-03-22T14:56:51Z)
• Hyperbolic Contrastive Learning [12.170564544949308]
  We propose a novel contrastive learning framework to learn semantic relationships in the hyperbolic space. We show that our proposed method achieves better results on self-supervised pretraining, supervised classification, and higher robust accuracy than baseline methods.
  arXiv  Detail & Related papers  (2023-02-02T20:47:45Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.