Semantic similarity prediction is better than other semantic similarity measures

• URL: http://arxiv.org/abs/2309.12697v2
• Date: Wed, 17 Jan 2024 08:50:59 GMT
• Title: Semantic similarity prediction is better than other semantic similarity measures
• Authors: Steffen Herbold
• Abstract summary: We argue that when we are only interested in measuring the semantic similarity, it is better to directly predict the similarity using a fine-tuned model for such a task. Using a fine-tuned model for the Semantic Textual Similarity Benchmark tasks (STS-B) from the GLUE benchmark, we define the STSScore approach and show that the resulting similarity is better aligned with our expectations on a robust semantic similarity measure than other approaches.
• Score: 5.176134438571082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic similarity between natural language texts is typically measured either by looking at the overlap between subsequences (e.g., BLEU) or by using embeddings (e.g., BERTScore, S-BERT). Within this paper, we argue that when we are only interested in measuring the semantic similarity, it is better to directly predict the similarity using a fine-tuned model for such a task. Using a fine-tuned model for the Semantic Textual Similarity Benchmark tasks (STS-B) from the GLUE benchmark, we define the STSScore approach and show that the resulting similarity is better aligned with our expectations on a robust semantic similarity measure than other approaches.

Related papers

• Predicting Text Preference Via Structured Comparative Reasoning [110.49560164568791]
  We introduce SC, a prompting approach that predicts text preferences by generating structured intermediate comparisons. We select consistent comparisons with a pairwise consistency comparator that ensures each aspect's comparisons clearly distinguish differences between texts. Our comprehensive evaluations across various NLP tasks, including summarization, retrieval, and automatic rating, demonstrate that SC equips LLMs to achieve state-of-the-art performance in text preference prediction.
  arXiv  Detail & Related papers  (2023-11-14T18:51:38Z)
• Description-Based Text Similarity [59.552704474862004]
  We identify the need to search for texts based on abstract descriptions of their content. We propose an alternative model that significantly improves when used in standard nearest neighbor search.
  arXiv  Detail & Related papers  (2023-05-21T17:14:31Z)
• ContraSim -- A Similarity Measure Based on Contrastive Learning [28.949004915740776]
  We develop a new similarity measure, dubbed ContraSim, based on contrastive learning. ContraSim learns a parameterized measure by using both similar and dissimilar examples. In all cases, ContraSim achieves much higher accuracy than previous similarity measures.
  arXiv  Detail & Related papers  (2023-03-29T19:43:26Z)
• Counting Like Human: Anthropoid Crowd Counting on Modeling the Similarity of Objects [92.80955339180119]
  mainstream crowd counting methods regress density map and integrate it to obtain counting results. Inspired by this, we propose a rational and anthropoid crowd counting framework.
  arXiv  Detail & Related papers  (2022-12-02T07:00:53Z)
• Identifying Ambiguous Similarity Conditions via Semantic Matching [49.06931755266372]
  We introduce Weakly Supervised Conditional Similarity Learning (WS-CSL) WS-CSL learns multiple embeddings to match semantic conditions without explicit condition labels such as "can fly" We propose the Distance Induced Semantic COndition VERification Network (DiscoverNet), which characterizes the instance-instance and triplets-condition relations in a "decompose-and-fuse" manner.
  arXiv  Detail & Related papers  (2022-04-08T13:15:55Z)
• Comparing in context: Improving cosine similarity measures with a metric tensor [0.0]
  Cosine similarity is a widely used measure of the relatedness of pre-trained word embeddings, trained on a language modeling goal. We propose instead the use of an extended cosine similarity measure to improve performance on that task, with gains in interpretability. We learn contextualized metrics and compare the results with the baseline values obtained using the standard cosine similarity measure, which consistently shows improvement. We also train a contextualized similarity measure for both SimLex-999 and WordSim-353, comparing the results with the corresponding baselines, and using these datasets as independent test sets for the all-context similarity measure learned on
  arXiv  Detail & Related papers  (2022-03-28T18:04:26Z)
• Attributable Visual Similarity Learning [90.69718495533144]
  This paper proposes an attributable visual similarity learning (AVSL) framework for a more accurate and explainable similarity measure between images. Motivated by the human semantic similarity cognition, we propose a generalized similarity learning paradigm to represent the similarity between two images with a graph. Experiments on the CUB-200-2011, Cars196, and Stanford Online Products datasets demonstrate significant improvements over existing deep similarity learning methods.
  arXiv  Detail & Related papers  (2022-03-28T17:35:31Z)
• Duality-Induced Regularizer for Semantic Matching Knowledge Graph Embeddings [70.390286614242]
  We propose a novel regularizer -- namely, DUality-induced RegulArizer (DURA) -- which effectively encourages the entities with similar semantics to have similar embeddings. Experiments demonstrate that DURA consistently and significantly improves the performance of state-of-the-art semantic matching models.
  arXiv  Detail & Related papers  (2022-03-24T09:24:39Z)
• MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity [0.0]
  Similarity is a comparative-subjective measure that varies with the domain within which it is considered. This paper presents a multi-layered semantic similarity network model built upon multiple similarity measures. It is shown to have demonstrated better performance scores in assessing sentence similarity.
  arXiv  Detail & Related papers  (2021-11-09T20:43:18Z)
• Semantic Answer Similarity for Evaluating Question Answering Models [2.279676596857721]
  SAS is a cross-encoder-based metric for the estimation of semantic answer similarity. We show that semantic similarity metrics based on recent transformer models correlate much better with human judgment than traditional lexical similarity metrics.
  arXiv  Detail & Related papers  (2021-08-13T09:12:27Z)
• Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures [0.0]
  We learn a categorical variational autoencoder based on off-the-shelf pre-trained language models. We empirically demonstrate that these finetuned models could be further improved by Latte-Mix.
  arXiv  Detail & Related papers  (2020-10-21T23:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.