Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model

• URL: http://arxiv.org/abs/2203.15595v1
• Date: Tue, 29 Mar 2022 14:04:53 GMT
• Title: Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model
• Authors: Benzhi Wang, Meiyu Liang, Feifei Kou and Mingying Xu
• Abstract summary: This paper proposes a cross-media scientific research achievements retrieval method based on deep language model (CARDL) It achieves a unified cross-media semantic representation by learning the semantic association between different modal data. Cross-media retrieval is realized through semantic similarity matching between different modal data.
• Score: 2.900289363118179
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Science and technology big data contain a lot of cross-media information.There are images and texts in the scientific paper.The s ingle modal search method cannot well meet the needs of scientific researchers.This paper proposes a cross-media scientific research achievements retrieval method based on deep language model (CARDL).It achieves a unified cross-media semantic representation by learning the semantic association between different modal data, and is applied to the generation of text semantic vector of scientific research achievements, and then cross-media retrieval is realized through semantic similarity matching between different modal data.Experimental results show that the proposed CARDL method achieves better cross-modal retrieval performance than existing methods. Key words science and technology big data ; cross-media retrieval; cross-media semantic association learning; deep language model; semantic similarity

Related papers

• VectorSearch: Enhancing Document Retrieval with Semantic Embeddings and Optimized Search [1.0411820336052784]
  We propose VectorSearch, which leverages advanced algorithms, embeddings, and indexing techniques for refined retrieval. By utilizing innovative multi-vector search operations and encoding searches with advanced language models, our approach significantly improves retrieval accuracy. Experiments on real-world datasets show that VectorSearch outperforms baseline metrics.
  arXiv  Detail & Related papers  (2024-09-25T21:58:08Z)
• Multi-Modal Retrieval For Large Language Model Based Speech Recognition [15.494654232953678]
  We propose multi-modal retrieval with two approaches: kNN-LM and cross-attention techniques. We show that speech-based multi-modal retrieval outperforms text based retrieval. We achieve state-of-the-art recognition results on the Spoken-Squad question answering dataset.
  arXiv  Detail & Related papers  (2024-06-13T22:55:22Z)
• Text-Video Retrieval with Global-Local Semantic Consistent Learning [122.15339128463715]
  We propose a simple yet effective method, Global-Local Semantic Consistent Learning (GLSCL) GLSCL capitalizes on latent shared semantics across modalities for text-video retrieval. Our method achieves comparable performance with SOTA as well as being nearly 220 times faster in terms of computational cost.
  arXiv  Detail & Related papers  (2024-05-21T11:59:36Z)
• Knowledge-Enhanced Hierarchical Information Correlation Learning for Multi-Modal Rumor Detection [82.94413676131545]
  We propose a novel knowledge-enhanced hierarchical information correlation learning approach (KhiCL) for multi-modal rumor detection. KhiCL exploits cross-modal joint dictionary to transfer the heterogeneous unimodality features into the common feature space. It extracts visual and textual entities from images and text, and designs a knowledge relevance reasoning strategy.
  arXiv  Detail & Related papers  (2023-06-28T06:08:20Z)
• Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis [89.04041100520881]
  This research proposes to retrieve textual and visual evidence based on the object, sentence, and whole image. We develop a novel approach to synthesize the object-level, image-level, and sentence-level information for better reasoning between the same and different modalities.
  arXiv  Detail & Related papers  (2023-05-25T15:26:13Z)
• Scientific and Technological Information Oriented Semantics-adversarial and Media-adversarial Cross-media Retrieval [21.630525836722036]
  Cross-media scientific and technological information retrieval is one of the important tasks in the cross-media study. We propose a scientific and technological information oriented Semantics-adversarial and Media-adversarial Cross-media Retrieval method (SMCR) to find an effective common subspace. SMCR minimizes the loss of inter-media semantic consistency in addition to modeling intra-media semantic discrimination, to preserve semantic similarity before and after mapping.
  arXiv  Detail & Related papers  (2022-03-16T13:31:48Z)
• Semantic Search for Large Scale Clinical Ontologies [63.71950996116403]
  We present a deep learning approach to build a search system for large clinical vocabularies. We propose a Triplet-BERT model and a method that generates training data based on semantic training data. The model is evaluated using five real benchmark data sets and the results show that our approach achieves high results on both free text to concept and concept to searching concept vocabularies.
  arXiv  Detail & Related papers  (2022-01-01T05:15:42Z)
• Semantic maps and metrics for science Semantic maps and metrics for science using deep transformer encoders [1.599072005190786]
  Recent advances in natural language understanding driven by deep transformer networks offer new possibilities for mapping science. Transformer embedding models capture shades of association and connotation that vary across different linguistic contexts. We report a procedure for encoding scientific documents with these tools, measuring their improvement over static word embeddings.
  arXiv  Detail & Related papers  (2021-04-13T04:12:20Z)
• Named Entity Recognition for Social Media Texts with Semantic Augmentation [70.44281443975554]
  Existing approaches for named entity recognition suffer from data sparsity problems when conducted on short and informal texts. We propose a neural-based approach to NER for social media texts where both local (from running text) and augmented semantics are taken into account.
  arXiv  Detail & Related papers  (2020-10-29T10:06:46Z)
• Deep Learning Techniques for Future Intelligent Cross-Media Retrieval [58.20547387332133]
  Cross-media retrieval plays a significant role in big data applications. We provide a novel taxonomy according to the challenges faced by multi-modal deep learning approaches. We present some well-known cross-media datasets used for retrieval.
  arXiv  Detail & Related papers  (2020-07-21T09:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.