Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders

• URL: http://arxiv.org/abs/2210.05302v1
• Date: Tue, 11 Oct 2022 09:52:52 GMT
• Title: Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders
• Authors: Qiwei Peng, David Weir, Julie Weeds
• Abstract summary: We propose to combine sentence encoders with an alignment component by representing each sentence as a list of predicate-argument spans. Empirical results show that the alignment component brings in both improved performance and interpretability for various sentence encoders.
• Score: 4.254099382808598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Previous works have demonstrated the effectiveness of utilising pre-trained sentence encoders based on their sentence representations for meaning comparison tasks. Though such representations are shown to capture hidden syntax structures, the direct similarity comparison between them exhibits weak sensitivity to word order and structural differences in given sentences. A single similarity score further makes the comparison process hard to interpret. Therefore, we here propose to combine sentence encoders with an alignment component by representing each sentence as a list of predicate-argument spans (where their span representations are derived from sentence encoders), and decomposing the sentence-level meaning comparison into the alignment between their spans for paraphrase identification tasks. Empirical results show that the alignment component brings in both improved performance and interpretability for various sentence encoders. After closer investigation, the proposed approach indicates increased sensitivity to structural difference and enhanced ability to distinguish non-paraphrases with high lexical overlap.

Related papers

• Predicting Text Preference Via Structured Comparative Reasoning [110.49560164568791]
  We introduce SC, a prompting approach that predicts text preferences by generating structured intermediate comparisons. We select consistent comparisons with a pairwise consistency comparator that ensures each aspect's comparisons clearly distinguish differences between texts. Our comprehensive evaluations across various NLP tasks, including summarization, retrieval, and automatic rating, demonstrate that SC equips LLMs to achieve state-of-the-art performance in text preference prediction.
  arXiv  Detail & Related papers  (2023-11-14T18:51:38Z)
• Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations [102.05351905494277]
  Sub-sentence encoder is a contrastively-learned contextual embedding model for fine-grained semantic representation of text. We show that sub-sentence encoders keep the same level of inference cost and space complexity compared to sentence encoders.
  arXiv  Detail & Related papers  (2023-11-07T20:38:30Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations [80.45474362071236]
  It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space. We propose InterSent, an end-to-end framework for learning interpretable sentence embeddings.
  arXiv  Detail & Related papers  (2023-05-24T00:44:49Z)
• Representation biases in sentence transformers [1.2129015549576372]
  We show that SOTA sentence transformers have a strong nominal-participant-set bias. C cosine similarities between pairs of sentences are more strongly determined by the overlap in the set of their noun participants.
  arXiv  Detail & Related papers  (2023-01-30T16:35:23Z)
• A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings [28.046786376565123]
  We propose a semantics-aware contrastive learning framework for sentence embeddings, termed Pseudo-Token BERT (PT-BERT) We exploit the pseudo-token space (i.e., latent semantic space) representation of a sentence while eliminating the impact of superficial features such as sentence length and syntax. Our model outperforms the state-of-the-art baselines on six standard semantic textual similarity (STS) tasks.
  arXiv  Detail & Related papers  (2022-03-11T12:29:22Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• A Self-supervised Representation Learning of Sentence Structure for Authorship Attribution [3.5991811164452923]
  We propose a self-supervised framework for learning structural representations of sentences. We evaluate the learned structural representations of sentences using different probing tasks, and subsequently utilize them in the authorship attribution task.
  arXiv  Detail & Related papers  (2020-10-14T02:57:10Z)
• Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
  We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
  arXiv  Detail & Related papers  (2020-10-11T15:13:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.