BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts

• URL: http://arxiv.org/abs/2410.23583v1
• Date: Thu, 31 Oct 2024 02:51:56 GMT
• Title: BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts
• Authors: Farshad Noravesh,
• Abstract summary: State-of-the-art models for relation extraction (RE) in the biomedical domain may suffer from the anisotropy problem. Contrastive learning methods can reduce this anisotropy phenomena, and also help to avoid class collapse in any classification problem. BioNCERE uses transfer learning and non-contrastive learning to avoid full or dimensional collapse as well as bypass overfitting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State-of-the-art models for relation extraction (RE) in the biomedical domain consider finetuning BioBERT using classification, but they may suffer from the anisotropy problem. Contrastive learning methods can reduce this anisotropy phenomena, and also help to avoid class collapse in any classification problem. In the present paper, a new training method called biological non-contrastive relation extraction (BioNCERE) is introduced for relation extraction without using any named entity labels for training to reduce annotation costs. BioNCERE uses transfer learning and non-contrastive learning to avoid full or dimensional collapse as well as bypass overfitting. It resolves RE in three stages by leveraging transfer learning two times. By freezing the weights learned in previous stages in the proposed pipeline and by leveraging non-contrastive learning in the second stage, the model predicts relations without any knowledge of named entities. Experiments have been done on SemMedDB that are almost similar to State-of-the-art performance on RE without using the information of named entities.

Related papers

• WHERE and WHICH: Iterative Debate for Biomedical Synthetic Data Augmentation [17.956185532113]
  This paper proposes a biomedical-dedicated rationale-based synthetic data augmentation method. Specific bio-relation similarity is measured to hold the augmented instance having a strong correlation with bio-relation. We evaluate our method on the BLURB and BigBIO benchmark, which includes 9 common datasets spanning four major BioNLP tasks.
  arXiv  Detail & Related papers  (2025-03-31T02:36:30Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Graph Relation Distillation for Efficient Biomedical Instance Segmentation [80.51124447333493]
  We propose a graph relation distillation approach for efficient biomedical instance segmentation. We introduce two graph distillation schemes deployed at both the intra-image level and the inter-image level. Experimental results on a number of biomedical datasets validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-01-12T04:41:23Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• ReOnto: A Neuro-Symbolic Approach for Biomedical Relation Extraction [3.263873198567265]
  ReOnto Relation Extraction (RE) is the task of extracting semantic relationships between entities in a sentence and aligning them to relations defined in a vocabulary. We present a novel technique that makes use of neuro symbolic knowledge for the RE task. Experimental results on two public biomedical datasets, BioRel and ADE, show that our method outperforms all the baselines.
  arXiv  Detail & Related papers  (2023-09-04T05:36:58Z)
• Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction? [39.17663642263077]
  Two key obstacles in biomedical relation extraction (RE) are the scarcity of annotations and the prevalence of instances without explicitly pre-defined labels. Existing approaches, which treat biomedical RE as a multi-class classification task, often result in poor generalization in low-resource settings. We present NBR, which converts biomedical RE as natural language inference formulation through indirect supervision.
  arXiv  Detail & Related papers  (2022-12-21T05:49:08Z)
• Joint Learning-based Causal Relation Extraction from Biomedical Literature [15.92139180835277]
  Experimental results on the BioCreative-V Track 4 corpus show that our joint learning model outperforms the separate models in BEL statement extraction. This demonstrates that our joint learning system reaches the state-of-the-art performance in Stage 2 compared with other systems.
  arXiv  Detail & Related papers  (2022-08-02T08:54:57Z)
• Does constituency analysis enhance domain-specific pre-trained BERT models for relation extraction? [0.0]
  The DrugProt track at BioCreative VII provides a manually-annotated corpus for the development and evaluation of relation extraction systems. We describe the ensemble system that we used for our submission, which combines predictions of fine-tuned bioBERT, sciBERT and const-bioBERT models by majority voting.
  arXiv  Detail & Related papers  (2021-11-25T10:27:10Z)
• Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction [52.63803634033647]
  Low-resource relation Extraction (LRE) aims to extract relation facts from limited labeled corpora when human annotation is scarce. We develop a Gradient Imitation Reinforcement Learning method to encourage pseudo label data to imitate the gradient descent direction on labeled data. We also propose a framework called GradLRE, which handles two major scenarios in low-resource relation extraction.
  arXiv  Detail & Related papers  (2021-09-14T03:51:15Z)
• Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching [58.26619456972598]
  Federated learning (FL) has emerged with increasing popularity to collaborate distributed medical institutions for training deep networks. This paper studies a practical yet challenging FL problem, named textitFederated Semi-supervised Learning (FSSL) We present a novel approach for this problem, which improves over traditional consistency regularization mechanism with a new inter-client relation matching scheme.
  arXiv  Detail & Related papers  (2021-06-16T07:58:00Z)
• ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Medical Image Segmentation [99.90263375737362]
  We propose ATSO, an asynchronous version of teacher-student optimization. ATSO partitions the unlabeled data into two subsets and alternately uses one subset to fine-tune the model and updates the label on the other subset. We evaluate ATSO on two popular medical image segmentation datasets and show its superior performance in various semi-supervised settings.
  arXiv  Detail & Related papers  (2020-06-24T04:05:12Z)
• A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction [2.771933807499954]
  We extend an entity-enriched relation classification BERT model to the problem of multiple instance learning. We define a simple data encoding scheme that significantly reduces noise. Our approach further encodes knowledge about the direction of relation triples, allowing for increased focus on relation learning.
  arXiv  Detail & Related papers  (2020-05-26T08:15:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.