AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledge

• URL: http://arxiv.org/abs/2307.07851v5
• Date: Sun, 24 Sep 2023 20:35:52 GMT
• Title: AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledge
• Authors: Tim Schopf, Emanuel Gerber, Malte Ostendorff, Florian Matthes
• Abstract summary: We present AspectCSE, an approach for aspect-based contrastive learning of sentence embeddings. We demonstrate that multi-aspect embeddings outperform single-aspect embeddings on aspect-specific information retrieval tasks.
• Score: 4.563449647618151
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generic sentence embeddings provide a coarse-grained approximation of semantic textual similarity but ignore specific aspects that make texts similar. Conversely, aspect-based sentence embeddings provide similarities between texts based on certain predefined aspects. Thus, similarity predictions of texts are more targeted to specific requirements and more easily explainable. In this paper, we present AspectCSE, an approach for aspect-based contrastive learning of sentence embeddings. Results indicate that AspectCSE achieves an average improvement of 3.97% on information retrieval tasks across multiple aspects compared to the previous best results. We also propose using Wikidata knowledge graph properties to train models of multi-aspect sentence embeddings in which multiple specific aspects are simultaneously considered during similarity predictions. We demonstrate that multi-aspect embeddings outperform single-aspect embeddings on aspect-specific information retrieval tasks. Finally, we examine the aspect-based sentence embedding space and demonstrate that embeddings of semantically similar aspect labels are often close, even without explicit similarity training between different aspect labels.

Related papers

• Predicting Text Preference Via Structured Comparative Reasoning [110.49560164568791]
  We introduce SC, a prompting approach that predicts text preferences by generating structured intermediate comparisons. We select consistent comparisons with a pairwise consistency comparator that ensures each aspect's comparisons clearly distinguish differences between texts. Our comprehensive evaluations across various NLP tasks, including summarization, retrieval, and automatic rating, demonstrate that SC equips LLMs to achieve state-of-the-art performance in text preference prediction.
  arXiv  Detail & Related papers  (2023-11-14T18:51:38Z)
• Aspect-oriented Opinion Alignment Network for Aspect-Based Sentiment Classification [14.212306015270208]
  We propose a novel Aspect-oriented Opinion Alignment Network (AOAN) to capture the contextual association between opinion words and the corresponding aspect. In addition, we design a multi-perspective attention mechanism that align relevant opinion information with respect to the given aspect. Our model achieves state-of-the-art results on three benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-22T13:55:36Z)
• Composition-contrastive Learning for Sentence Embeddings [23.85590618900386]
  This work is the first to do so without incurring costs in auxiliary training objectives or additional network parameters. Experimental results on semantic textual similarity tasks show improvements over baselines that are comparable with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-07-14T14:39:35Z)
• Relational Sentence Embedding for Flexible Semantic Matching [86.21393054423355]
  We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings. RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
  arXiv  Detail & Related papers  (2022-12-17T05:25:17Z)
• Interpreting BERT-based Text Similarity via Activation and Saliency Maps [26.279593839644836]
  We present an unsupervised technique for explaining paragraph similarities inferred by pre-trained BERT models. By looking at a pair of paragraphs, our technique identifies important words that dictate each paragraph's semantics, matches between the words in both paragraphs, and retrieves the most important pairs that explain the similarity between the two.
  arXiv  Detail & Related papers  (2022-08-13T10:06:24Z)
• MCSE: Multimodal Contrastive Learning of Sentence Embeddings [23.630041603311923]
  We propose a sentence embedding learning approach that exploits both visual and textual information via a multimodal contrastive objective. We show that our approach consistently improves the performance across various datasets and pre-trained encoders.
  arXiv  Detail & Related papers  (2022-04-22T21:19:24Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)
• A Comparative Study on Structural and Semantic Properties of Sentence Embeddings [77.34726150561087]
  We propose a set of experiments using a widely-used large-scale data set for relation extraction. We show that different embedding spaces have different degrees of strength for the structural and semantic properties. These results provide useful information for developing embedding-based relation extraction methods.
  arXiv  Detail & Related papers  (2020-09-23T15:45:32Z)
• Consensus-Aware Visual-Semantic Embedding for Image-Text Matching [69.34076386926984]
  Image-text matching plays a central role in bridging vision and language. Most existing approaches only rely on the image-text instance pair to learn their representations. We propose a Consensus-aware Visual-Semantic Embedding model to incorporate the consensus information.
  arXiv  Detail & Related papers  (2020-07-17T10:22:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.