R$^2$-Net: Relation of Relation Learning Network for Sentence Semantic
Matching
- URL: http://arxiv.org/abs/2012.08920v1
- Date: Wed, 16 Dec 2020 13:11:30 GMT
- Title: R$^2$-Net: Relation of Relation Learning Network for Sentence Semantic
Matching
- Authors: Kun Zhang, Le Wu, Guangyi Lv, Meng Wang, Enhong Chen, Shulan Ruan
- Abstract summary: We propose a Relation of Relation Learning Network (R2-Net) for sentence semantic matching.
We first employ BERT to encode the input sentences from a global perspective.
Then a CNN-based encoder is designed to capture keywords and phrase information from a local perspective.
To fully leverage labels for better relation information extraction, we introduce a self-supervised relation of relation classification task.
- Score: 58.72111690643359
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Sentence semantic matching is one of the fundamental tasks in natural
language processing, which requires an agent to determine the semantic relation
among input sentences. Recently, deep neural networks have achieved impressive
performance in this area, especially BERT. Despite the effectiveness of these
models, most of them treat output labels as meaningless one-hot vectors,
underestimating the semantic information and guidance of relations that these
labels reveal, especially for tasks with a small number of labels. To address
this problem, we propose a Relation of Relation Learning Network (R2-Net) for
sentence semantic matching. Specifically, we first employ BERT to encode the
input sentences from a global perspective. Then a CNN-based encoder is designed
to capture keywords and phrase information from a local perspective. To fully
leverage labels for better relation information extraction, we introduce a
self-supervised relation of relation classification task for guiding R2-Net to
consider more about labels. Meanwhile, a triplet loss is employed to
distinguish the intra-class and inter-class relations in a finer granularity.
Empirical experiments on two sentence semantic matching tasks demonstrate the
superiority of our proposed model. As a byproduct, we have released the codes
to facilitate other researches.
Related papers
- Multi-Label Classification for Implicit Discourse Relation Recognition [10.280148603465697]
We explore various multi-label classification frameworks to handle implicit discourse relation recognition.
We show that multi-label classification methods don't depress performance for single-label prediction.
arXiv Detail & Related papers (2024-06-06T19:37:25Z) - NP$^2$L: Negative Pseudo Partial Labels Extraction for Graph Neural
Networks [48.39834063008816]
Pseudo labels are used in graph neural networks (GNNs) to assist learning at the message-passing level.
In this paper, we introduce a new method to use pseudo labels in GNNs.
We show that our method is more accurate if they are selected by not overlapping partial labels and defined as negative node pairs relations.
arXiv Detail & Related papers (2023-10-02T11:13:59Z) - Description-Enhanced Label Embedding Contrastive Learning for Text
Classification [65.01077813330559]
Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task.
Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets.
external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
arXiv Detail & Related papers (2023-06-15T02:19:34Z) - Towards Unsupervised Recognition of Token-level Semantic Differences in
Related Documents [61.63208012250885]
We formulate recognizing semantic differences as a token-level regression task.
We study three unsupervised approaches that rely on a masked language model.
Our results show that an approach based on word alignment and sentence-level contrastive learning has a robust correlation to gold labels.
arXiv Detail & Related papers (2023-05-22T17:58:04Z) - Disambiguation of Company names via Deep Recurrent Networks [101.90357454833845]
We propose a Siamese LSTM Network approach to extract -- via supervised learning -- an embedding of company name strings.
We analyse how an Active Learning approach to prioritise the samples to be labelled leads to a more efficient overall learning pipeline.
arXiv Detail & Related papers (2023-03-07T15:07:57Z) - Relational Sentence Embedding for Flexible Semantic Matching [86.21393054423355]
We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings.
RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
arXiv Detail & Related papers (2022-12-17T05:25:17Z) - Group is better than individual: Exploiting Label Topologies and Label
Relations for Joint Multiple Intent Detection and Slot Filling [39.76268402567324]
We construct a Heterogeneous Label Graph (HLG) containing two kinds of topologies.
Label correlations are leveraged to enhance semantic-label interactions.
We also propose the label-aware inter-dependent decoding mechanism to further exploit the label correlations for decoding.
arXiv Detail & Related papers (2022-10-19T08:21:43Z) - DEIM: An effective deep encoding and interaction model for sentence
matching [0.0]
We propose a sentence matching method based on deep encoding and interaction to extract deep semantic information.
In the encoder layer,we refer to the information of another sentence in the process of encoding a single sentence, and later use a algorithm to fuse the information.
In the interaction layer, we use a bidirectional attention mechanism and a self-attention mechanism to obtain deep semantic information.
arXiv Detail & Related papers (2022-03-20T07:59:42Z) - DLDL: Dynamic Label Dictionary Learning via Hypergraph Regularization [17.34373273007931]
We propose a Dynamic Label Dictionary Learning (DLDL) algorithm to generate the soft label matrix for unlabeled data.
Specifically, we employ hypergraph manifold regularization to keep the relations among original data, transformed data, and soft labels consistent.
arXiv Detail & Related papers (2020-10-23T14:07:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.