Efficient Sentence Embedding via Semantic Subspace Analysis
- URL: http://arxiv.org/abs/2002.09620v2
- Date: Wed, 4 Mar 2020 04:49:45 GMT
- Title: Efficient Sentence Embedding via Semantic Subspace Analysis
- Authors: Bin Wang and Fenxiao Chen and Yuncheng Wang and C.-C. Jay Kuo
- Abstract summary: We develop a sentence representation scheme by analyzing semantic subspaces of constituent words.
Experimental results show that it offers comparable or better performance than the state-of-the-art.
- Score: 33.44637608270928
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: A novel sentence embedding method built upon semantic subspace analysis,
called semantic subspace sentence embedding (S3E), is proposed in this work.
Given the fact that word embeddings can capture semantic relationship while
semantically similar words tend to form semantic groups in a high-dimensional
embedding space, we develop a sentence representation scheme by analyzing
semantic subspaces of its constituent words. Specifically, we construct a
sentence model from two aspects. First, we represent words that lie in the same
semantic group using the intra-group descriptor. Second, we characterize the
interaction between multiple semantic groups with the inter-group descriptor.
The proposed S3E method is evaluated on both textual similarity tasks and
supervised tasks. Experimental results show that it offers comparable or better
performance than the state-of-the-art. The complexity of our S3E method is also
much lower than other parameterized models.
Related papers
- Self-Supervised Speech Representations are More Phonetic than Semantic [52.02626675137819]
Self-supervised speech models (S3Ms) have become an effective backbone for speech applications.
We seek a more fine-grained analysis of the word-level linguistic properties encoded in S3Ms.
Our study reveals that S3M representations consistently and significantly exhibit more phonetic than semantic similarity.
arXiv Detail & Related papers (2024-06-12T20:04:44Z) - GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understanding [66.5538429726564]
Self-supervised 3D representation learning aims to learn effective representations from large-scale unlabeled point clouds.
We propose GroupContrast, a novel approach that combines segment grouping and semantic-aware contrastive learning.
arXiv Detail & Related papers (2024-03-14T17:59:59Z) - A Comparative Study of Sentence Embedding Models for Assessing Semantic
Variation [0.0]
We compare several recent sentence embedding methods via time-series of semantic similarity between successive sentences and matrices of pairwise sentence similarity for multiple books of literature.
We find that most of the sentence embedding methods considered do infer highly correlated patterns of semantic similarity in a given document, but show interesting differences.
arXiv Detail & Related papers (2023-08-08T23:31:10Z) - Bridging Continuous and Discrete Spaces: Interpretable Sentence
Representation Learning via Compositional Operations [80.45474362071236]
It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space.
We propose InterSent, an end-to-end framework for learning interpretable sentence embeddings.
arXiv Detail & Related papers (2023-05-24T00:44:49Z) - Relational Sentence Embedding for Flexible Semantic Matching [86.21393054423355]
We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings.
RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
arXiv Detail & Related papers (2022-12-17T05:25:17Z) - Retrofitting Multilingual Sentence Embeddings with Abstract Meaning
Representation [70.58243648754507]
We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR)
Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously.
Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic similarity and transfer tasks.
arXiv Detail & Related papers (2022-10-18T11:37:36Z) - Lexical semantics enhanced neural word embeddings [4.040491121427623]
hierarchy-fitting is a novel approach to modelling semantic similarity nuances inherently stored in the IS-A hierarchies.
Results demonstrate the efficacy of hierarchy-fitting in specialising neural embeddings with semantic relations in late fusion.
arXiv Detail & Related papers (2022-10-03T08:10:23Z) - A bilingual approach to specialised adjectives through word embeddings
in the karstology domain [3.92181732547846]
We present an experiment in extracting adjectives which express a specific semantic relation using word embeddings.
The results of the experiment are then thoroughly analysed and categorised into groups of adjectives exhibiting formal or semantic similarity.
arXiv Detail & Related papers (2022-03-31T08:27:15Z) - Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation.
This paper aims to address the issue with a mask-and-predict strategy.
We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions.
Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
arXiv Detail & Related papers (2021-10-04T03:59:15Z) - Semantic and Syntactic Enhanced Aspect Sentiment Triplet Extraction [18.331779474247323]
Aspect Sentiment Triplet Extraction aims to extract triplets from sentences, where each triplet includes an entity, its associated sentiment, and the opinion span explaining the reason for the sentiment.
We propose a Semantic and Syntactic Enhanced aspect Sentiment triplet Extraction model (S3E2) to fully exploit the syntactic and semantic relationships between the triplet elements and jointly extract them.
arXiv Detail & Related papers (2021-06-07T03:16:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.