Relational Sentence Embedding for Flexible Semantic Matching

• URL: http://arxiv.org/abs/2212.08802v2
• Date: Thu, 8 Jun 2023 12:44:28 GMT
• Title: Relational Sentence Embedding for Flexible Semantic Matching
• Authors: Bin Wang, Haizhou Li
• Abstract summary: We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings. RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
• Score: 86.21393054423355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Relational Sentence Embedding (RSE), a new paradigm to further discover the potential of sentence embeddings. Prior work mainly models the similarity between sentences based on their embedding distance. Because of the complex semantic meanings conveyed, sentence pairs can have various relation types, including but not limited to entailment, paraphrasing, and question-answer. It poses challenges to existing embedding methods to capture such relational information. We handle the problem by learning associated relational embeddings. Specifically, a relation-wise translation operation is applied to the source sentence to infer the corresponding target sentence with a pre-trained Siamese-based encoder. The fine-grained relational similarity scores can be computed from learned embeddings. We benchmark our method on 19 datasets covering a wide range of tasks, including semantic textual similarity, transfer, and domain-specific tasks. Experimental results show that our method is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art sentence embedding methods. https://github.com/BinWang28/RSE

Related papers

• A Comparative Study of Sentence Embedding Models for Assessing Semantic Variation [0.0]
  We compare several recent sentence embedding methods via time-series of semantic similarity between successive sentences and matrices of pairwise sentence similarity for multiple books of literature. We find that most of the sentence embedding methods considered do infer highly correlated patterns of semantic similarity in a given document, but show interesting differences.
  arXiv  Detail & Related papers  (2023-08-08T23:31:10Z)
• Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations [80.45474362071236]
  It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space. We propose InterSent, an end-to-end framework for learning interpretable sentence embeddings.
  arXiv  Detail & Related papers  (2023-05-24T00:44:49Z)
• Sentence Representations via Gaussian Embedding [15.235687410343171]
  GaussCSE is a contrastive learning framework for sentence embedding. It can handle asymmetric relationships between sentences, along with a similarity measure for identifying inclusion relations. Our experiments show that GaussCSE achieves the same performance as previous methods in natural language inference tasks.
  arXiv  Detail & Related papers  (2023-05-22T12:51:38Z)
• Pre-trained Sentence Embeddings for Implicit Discourse Relation Classification [26.973476248983477]
  Implicit discourse relations bind smaller linguistic units into coherent texts. We explore the utility of pre-trained sentence embeddings as base representations in a neural network for implicit discourse relation sense classification.
  arXiv  Detail & Related papers  (2022-10-20T04:17:03Z)
• Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation [70.58243648754507]
  We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR) Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic similarity and transfer tasks.
  arXiv  Detail & Related papers  (2022-10-18T11:37:36Z)
• On the Sentence Embeddings from Pre-trained Language Models [78.45172445684126]
  In this paper, we argue that the semantic information in the BERT embeddings is not fully exploited. We find that BERT always induces a non-smooth anisotropic semantic space of sentences, which harms its performance of semantic similarity. We propose to transform the anisotropic sentence embedding distribution to a smooth and isotropic Gaussian distribution through normalizing flows that are learned with an unsupervised objective.
  arXiv  Detail & Related papers  (2020-11-02T13:14:57Z)
• A Comparative Study on Structural and Semantic Properties of Sentence Embeddings [77.34726150561087]
  We propose a set of experiments using a widely-used large-scale data set for relation extraction. We show that different embedding spaces have different degrees of strength for the structural and semantic properties. These results provide useful information for developing embedding-based relation extraction methods.
  arXiv  Detail & Related papers  (2020-09-23T15:45:32Z)
• Combining Word Embeddings and N-grams for Unsupervised Document Summarization [2.1591018627187286]
  Graph-based extractive document summarization relies on the quality of the sentence similarity graph. We employ off-the-shelf deep embedding features and tf-idf features, and introduce a new text similarity metric. Our approach can outperform the tf-idf based approach and achieve state-of-the-art performance on the DUC04 dataset.
  arXiv  Detail & Related papers  (2020-04-25T00:22:46Z)
• Multiplex Word Embeddings for Selectional Preference Acquisition [70.33531759861111]
  We propose a multiplex word embedding model, which can be easily extended according to various relations among words. Our model can effectively distinguish words with respect to different relations without introducing unnecessary sparseness.
  arXiv  Detail & Related papers  (2020-01-09T04:47:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.