Using External knowledge to Enhanced PLM for Semantic Matching

• URL: http://arxiv.org/abs/2505.06605v1
• Date: Sat, 10 May 2025 11:33:48 GMT
• Title: Using External knowledge to Enhanced PLM for Semantic Matching
• Authors: Min Li, Chun Yuan,
• Abstract summary: In this paper, we use external knowledge to enhance the pre-trained semantic relevance discrimination model.<n> Experimental results on 10 public datasets show that our method achieves consistent improvements in performance.
• Score: 38.125341836302525
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Modeling semantic relevance has always been a challenging and critical task in natural language processing. In recent years, with the emergence of massive amounts of annotated data, it has become feasible to train complex models, such as neural network-based reasoning models. These models have shown excellent performance in practical applications and have achieved the current state-ofthe-art performance. However, even with such large-scale annotated data, we still need to think: Can machines learn all the knowledge necessary to perform semantic relevance detection tasks based on this data alone? If not, how can neural network-based models incorporate external knowledge into themselves, and how can relevance detection models be constructed to make full use of external knowledge? In this paper, we use external knowledge to enhance the pre-trained semantic relevance discrimination model. Experimental results on 10 public datasets show that our method achieves consistent improvements in performance compared to the baseline model.

Related papers

• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Learning Latent Hardening (LLH): Enhancing Deep Learning with Domain Knowledge for Material Inverse Problems [0.0]
  In this study, the incorporation of domain-specific knowledge of the mechanical behavior of material microstructures is investigated.<n>To overcome data limitations, a two-step framework, Learning Latent Hardening (LLH) is proposed.<n>The results from the models with domain-specific information consistently achieved higher $R2$ values compared to models without prior knowledge.
  arXiv  Detail & Related papers  (2025-01-17T03:09:25Z)
• The Extrapolation Power of Implicit Models [2.3526338188342653]
  Implicit models are put to the test across various extrapolation scenarios: out-of-distribution, geographical, and temporal shifts. Our experiments consistently demonstrate significant performance advantage with implicit models.
  arXiv  Detail & Related papers  (2024-07-19T16:01:37Z)
• Machine Learning vs Deep Learning: The Generalization Problem [0.0]
  This study investigates the comparative abilities of traditional machine learning (ML) models and deep learning (DL) algorithms in terms of extrapolation. We present an empirical analysis where both ML and DL models are trained on an exponentially growing function and then tested on values outside the training domain. Our findings suggest that deep learning models possess inherent capabilities to generalize beyond the training scope.
  arXiv  Detail & Related papers  (2024-03-03T21:42:55Z)
• Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model [74.62272538148245]
  We show that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other. We investigate if it is possible to transfer such "complementary" knowledge from one model to another without performance degradation.
  arXiv  Detail & Related papers  (2023-10-26T17:59:46Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Striving for data-model efficiency: Identifying data externalities on group performance [75.17591306911015]
  Building trustworthy, effective, and responsible machine learning systems hinges on understanding how differences in training data and modeling decisions interact to impact predictive performance. We focus on a particular type of data-model inefficiency, in which adding training data from some sources can actually lower performance evaluated on key sub-groups of the population. Our results indicate that data-efficiency is a key component of both accurate and trustworthy machine learning.
  arXiv  Detail & Related papers  (2022-11-11T16:48:27Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering [17.7709632238066]
  Data artifacts incentivize machine learning models to learn non-transferable generalizations. We investigate whether popular datasets such as MultiWOZ contain such data artifacts. We propose a contrastive learning based framework to encourage the model to ignore these cues and focus on learning generalisable patterns.
  arXiv  Detail & Related papers  (2022-05-20T03:13:02Z)
• BERT WEAVER: Using WEight AVERaging to enable lifelong learning for transformer-based models in biomedical semantic search engines [49.75878234192369]
  We present WEAVER, a simple, yet efficient post-processing method that infuses old knowledge into the new model. We show that applying WEAVER in a sequential manner results in similar word embedding distributions as doing a combined training on all data at once.
  arXiv  Detail & Related papers  (2022-02-21T10:34:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.