Retrieval-guided Twin Fusion with Similarity-aware Contrast for Molecule-Text Alignment

• URL: http://arxiv.org/abs/2608.16005v1
• Date: Mon, 17 Aug 2026 01:52:54 GMT
• Title: Retrieval-guided Twin Fusion with Similarity-aware Contrast for Molecule-Text Alignment
• Abstract summary: We propose a novel approach named Retrieval-guided Twin Fusion with Similarity-aware Contrast (RISEN) for molecule-text alignment.<n>The core idea of RISEN is to construct a latent twin molecule for each substructure with cross-modal retrieval for semantic enhancement.<n>In particular, for each substructure query, we retrieve relevant textual descriptions and sample several molecules that share similar descriptions of substructures.
• Score: 16.818672826278327
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper studies the problem of molecule-text alignment, which aims to project molecules and their textual descriptions into a joint latent space for downstream tasks including molecule search and molecular property prediction. Previous approaches typically combine graph structure mining with contrastive learning to enhance joint representation learning. However, they typically neglect fine-grained semantic relationships between substructures and texts, leading to suboptimal performance on downstream tasks. Towards this end, we propose a novel approach named Retrieval-guided Twin Fusion with Similarity-aware Contrast (RISEN) for molecule-text alignment. The core idea of RISEN is to construct a latent twin molecule for each substructure with cross-modal retrieval for semantic enhancement. In particular, for each substructure query, we retrieve relevant textual descriptions and sample several molecules that share similar descriptions of substructures. Then, we aggregate their representations via attention pooling for a twin latent representation, which would be further fused with the original substructure for representation enrichment. In addition, we measure the similarity across substructures and texts, which would further guide cross-modal contrastive learning with soft thresholding. Extensive experiments on benchmark datasets validate the superiority of the proposed RISEN in comparison with existing baselines.

Related papers

• LumiX: Structured and Coherent Text-to-Intrinsic Generation [56.659456254026985]
  We present LumiX, a structured diffusion framework for coherent text-to-intrinsic generation.<n>LumiX produces coherent and physically meaningful results, achieving 23% higher alignment and a better preference score.<n>It can also perform image-conditioned decomposition within the same framework.
  arXiv  Detail & Related papers  (2025-12-02T13:56:02Z)
• RTMol: Rethinking Molecule-text Alignment in a Round-trip View [4.597922051722059]
  We propose RTMol, a bidirectional alignment framework that unifies molecular captioning and text-to-SMILES generation through self-supervised round-trip learning.<n>Experiments demonstrate that RTMol enhances bidirectional alignment performance by up to 47% across various LLMs.
  arXiv  Detail & Related papers  (2025-11-15T09:55:55Z)
• Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction [6.478238734128006]
  Relation extraction (RE) aims to identify semantic relations between entities in unstructured text.<n>underlineRetrieval underlineOver underlineClassification (ROC) is a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics.<n>ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning.
  arXiv  Detail & Related papers  (2025-09-25T13:38:38Z)
• StructCoh: Structured Contrastive Learning for Context-Aware Text Semantic Matching [10.000850856259866]
  StructCoh is a graph-enhanced contrastive learning framework.<n>A hierarchical contrastive objective enforces consistency at multiple granularities.<n>Experiments on three legal document matching benchmarks and academic plagiarism detection datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2025-09-02T07:21:36Z)
• Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval [24.061535843472427]
  We introduce the Optimal TRansport-based Multi-grained Alignments model (ORMA) ORMA is a novel approach that facilitates multi-grained alignments between textual descriptions and molecules. Experimental results on the ChEBI-20 and PCdes datasets demonstrate that ORMA significantly outperforms existing state-of-the-art (SOTA) models.
  arXiv  Detail & Related papers  (2024-11-04T06:30:52Z)
• Towards Cross-Modal Text-Molecule Retrieval with Better Modality Alignment [23.86314167123294]
  Cross-modal text-molecule retrieval model aims to learn a shared feature space of the text and molecule modalities for accurate similarity calculation. Previous works are inadequate in capturing modality-shared features considering the significant gap between text sequences and molecule graphs. We propose a novel cross-modal text-molecule retrieval model with two-fold improvements.
  arXiv  Detail & Related papers  (2024-10-31T08:03:13Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• Atomas: Hierarchical Alignment on Molecule-Text for Unified Molecule Understanding and Generation [42.08917809689811]
  Cross-modal representation learning has emerged as a promising direction for enhancing the quality of molecular representation.<n>We propose Atomas, a hierarchical molecular representation learning framework that jointly learns representations from SMILES strings and text.<n>Atomas achieves superior performance across 12 tasks on 11 datasets, outperforming 11 baseline models.
  arXiv  Detail & Related papers  (2024-04-23T12:35:44Z)
• DiffCloth: Diffusion Based Garment Synthesis and Manipulation via Structural Cross-modal Semantic Alignment [124.57488600605822]
  Cross-modal garment synthesis and manipulation will significantly benefit the way fashion designers generate garments. We introduce DiffCloth, a diffusion-based pipeline for cross-modal garment synthesis and manipulation. Experiments on the CM-Fashion benchmark demonstrate that DiffCloth both yields state-of-the-art garment synthesis results.
  arXiv  Detail & Related papers  (2023-08-22T05:43:33Z)
• Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis [89.04041100520881]
  This research proposes to retrieve textual and visual evidence based on the object, sentence, and whole image. We develop a novel approach to synthesize the object-level, image-level, and sentence-level information for better reasoning between the same and different modalities.
  arXiv  Detail & Related papers  (2023-05-25T15:26:13Z)
• Learnable Pillar-based Re-ranking for Image-Text Retrieval [119.9979224297237]
  Image-text retrieval aims to bridge the modality gap and retrieve cross-modal content based on semantic similarities. Re-ranking, a popular post-processing practice, has revealed the superiority of capturing neighbor relations in single-modality retrieval tasks. We propose a novel learnable pillar-based re-ranking paradigm for image-text retrieval.
  arXiv  Detail & Related papers  (2023-04-25T04:33:27Z)
• Relation Clustering in Narrative Knowledge Graphs [71.98234178455398]
  relational sentences in the original text are embedded (with SBERT) and clustered in order to merge together semantically similar relations. Preliminary tests show that such clustering might successfully detect similar relations, and provide a valuable preprocessing for semi-supervised approaches.
  arXiv  Detail & Related papers  (2020-11-27T10:43:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.