Improving Scholarly Knowledge Representation: Evaluating BERT-based
Models for Scientific Relation Classification
- URL: http://arxiv.org/abs/2004.06153v2
- Date: Mon, 13 Jul 2020 14:30:03 GMT
- Title: Improving Scholarly Knowledge Representation: Evaluating BERT-based
Models for Scientific Relation Classification
- Authors: Ming Jiang, Jennifer D'Souza, S\"oren Auer, J. Stephen Downie
- Abstract summary: We show that domain-specific pre-training corpus benefits the Bert-based classification model to identify type of scientific relations.
Although the strategy of predicting a single relation each time achieves a higher classification accuracy, the latter strategy demonstrates a more consistent performance in the corpus with either a large or small size of annotations.
- Score: 5.8962650619804755
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With the rapid growth of research publications, there is a vast amount of
scholarly knowledge that needs to be organized in digital libraries. To deal
with this challenge, techniques relying on knowledge-graph structures are being
advocated. Within such graph-based pipelines, inferring relation types between
related scientific concepts is a crucial step. Recently, advanced techniques
relying on language models pre-trained on the large corpus have been popularly
explored for automatic relation classification. Despite remarkable
contributions that have been made, many of these methods were evaluated under
different scenarios, which limits their comparability. To this end, we present
a thorough empirical evaluation on eight Bert-based classification models by
focusing on two key factors: 1) Bert model variants, and 2) classification
strategies. Experiments on three corpora show that domain-specific pre-training
corpus benefits the Bert-based classification model to identify the type of
scientific relations. Although the strategy of predicting a single relation
each time achieves a higher classification accuracy than the strategy of
identifying multiple relation types simultaneously in general, the latter
strategy demonstrates a more consistent performance in the corpus with either a
large or small size of annotations. Our study aims to offer recommendations to
the stakeholders of digital libraries for selecting the appropriate technique
to build knowledge-graph-based systems for enhanced scholarly information
organization.
Related papers
- Why do you cite? An investigation on citation intents and decision-making classification processes [1.7812428873698407]
This study emphasizes the importance of trustfully classifying citation intents.
We present a study utilizing advanced Ensemble Strategies for Citation Intent Classification (CIC)
One of our models sets as a new state-of-the-art (SOTA) with an 89.46% Macro-F1 score on the SciCite benchmark.
arXiv Detail & Related papers (2024-07-18T09:29:33Z) - Detecting Statements in Text: A Domain-Agnostic Few-Shot Solution [1.3654846342364308]
State-of-the-art approaches usually involve fine-tuning models on large annotated datasets, which are costly to produce.
We propose and release a qualitative and versatile few-shot learning methodology as a common paradigm for any claim-based textual classification task.
We illustrate this methodology in the context of three tasks: climate change contrarianism detection, topic/stance classification and depression-relates symptoms detection.
arXiv Detail & Related papers (2024-05-09T12:03:38Z) - Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones.
This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
arXiv Detail & Related papers (2024-01-29T18:27:52Z) - Evaluating BERT-based Scientific Relation Classifiers for Scholarly
Knowledge Graph Construction on Digital Library Collections [5.8962650619804755]
Inferring semantic relations between related scientific concepts is a crucial step.
BERT-based pre-trained models have been popularly explored for automatic relation classification.
Existing methods are primarily evaluated on clean texts.
To address these limitations, we started by creating OCR-noisy texts.
arXiv Detail & Related papers (2023-05-03T17:32:16Z) - Enhancing Identification of Structure Function of Academic Articles
Using Contextual Information [6.28532577139029]
This paper takes articles of the ACL conference as the corpus to identify the structure function of academic articles.
We employ the traditional machine learning models and deep learning models to construct the classifiers based on various feature input.
Inspired by (2), this paper introduces contextual information into the deep learning models and achieved significant results.
arXiv Detail & Related papers (2021-11-28T11:21:21Z) - Partner-Assisted Learning for Few-Shot Image Classification [54.66864961784989]
Few-shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation.
In this paper, we focus on the design of training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples.
We propose a two-stage training scheme, which first trains a partner encoder to model pair-wise similarities and extract features serving as soft-anchors, and then trains a main encoder by aligning its outputs with soft-anchors while attempting to maximize classification performance.
arXiv Detail & Related papers (2021-09-15T22:46:19Z) - ECKPN: Explicit Class Knowledge Propagation Network for Transductive
Few-shot Learning [53.09923823663554]
Class-level knowledge can be easily learned by humans from just a handful of samples.
We propose an Explicit Class Knowledge Propagation Network (ECKPN) to address this problem.
We conduct extensive experiments on four few-shot classification benchmarks, and the experimental results show that the proposed ECKPN significantly outperforms the state-of-the-art methods.
arXiv Detail & Related papers (2021-06-16T02:29:43Z) - Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
We investigate three schemes to improve the model generalization ability for few-shot settings.
We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data.
We create new state-of-the-art results on both few-shot and training-free settings.
arXiv Detail & Related papers (2020-12-29T23:43:16Z) - A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
We develop a list of diagnostic properties for evaluating existing explainability techniques.
We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
arXiv Detail & Related papers (2020-09-25T12:01:53Z) - A Survey on Text Classification: From Shallow to Deep Learning [83.47804123133719]
The last decade has seen a surge of research in this area due to the unprecedented success of deep learning.
This paper fills the gap by reviewing the state-of-the-art approaches from 1961 to 2021.
We create a taxonomy for text classification according to the text involved and the models used for feature extraction and classification.
arXiv Detail & Related papers (2020-08-02T00:09:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.