A Self-supervised Approach for Semantic Indexing in the Context of COVID-19 Pandemic

• URL: http://arxiv.org/abs/2010.03544v1
• Date: Wed, 7 Oct 2020 17:43:55 GMT
• Title: A Self-supervised Approach for Semantic Indexing in the Context of COVID-19 Pandemic
• Authors: Nima Ebadi, Peyman Najafirad
• Abstract summary: We present a novel semantic indexing approach based on the state-of-the-art self-supervised representation learning and transformer encoding. Our study sheds light on the main challenges confronting semantic indexing models during a pandemic, namely new domains and drastic changes of their distributions.
• Score: 1.8782750537161614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The pandemic has accelerated the pace at which COVID-19 scientific papers are published. In addition, the process of manually assigning semantic indexes to these papers by experts is even more time-consuming and overwhelming in the current health crisis. Therefore, there is an urgent need for automatic semantic indexing models which can effectively scale-up to newly introduced concepts and rapidly evolving distributions of the hyperfocused related literature. In this research, we present a novel semantic indexing approach based on the state-of-the-art self-supervised representation learning and transformer encoding exclusively suitable for pandemic crises. We present a case study on a novel dataset that is based on COVID-19 papers published and manually indexed in PubMed. Our study shows that our self-supervised model outperforms the best performing models of BioASQ Task 8a by micro-F1 score of 0.1 and LCA-F score of 0.08 on average. Our model also shows superior performance on detecting the supplementary concepts which is quite important when the focus of the literature has drastically shifted towards specific concepts related to the pandemic. Our study sheds light on the main challenges confronting semantic indexing models during a pandemic, namely new domains and drastic changes of their distributions, and as a superior alternative for such situations, propose a model founded on approaches which have shown auspicious performance in improving generalization and data efficiency in various NLP tasks. We also show the joint indexing of major Medical Subject Headings (MeSH) and supplementary concepts improves the overall performance.

Related papers

• Perturbation Ontology based Graph Attention Networks [26.95077612390953]
  Ontology-based Graph Attention Networks (POGAT) is a novel methodology that combines ontology subgraphs with an advanced self-supervised learning paradigm to achieve a deep contextual understanding. POGAT significantly outperforms state-of-the-art baselines, achieving a groundbreaking improvement of up to 10.78% in F1-score for the critical task of link prediction and 12.01% in Micro-F1 for the critical task of node classification.
  arXiv  Detail & Related papers  (2024-11-27T17:12:14Z)
• Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation [56.87049651707208]
  Few-shot Semantic has evolved into In-context tasks, morphing into a crucial element in assessing generalist segmentation models. Our initial focus lies in understanding how to facilitate interaction between the query image and the support image, resulting in the proposal of a KV fusion method within the self-attention framework. Based on our analysis, we establish a simple and effective framework named DiffewS, maximally retaining the original Latent Diffusion Model's generative framework.
  arXiv  Detail & Related papers  (2024-10-03T10:33:49Z)
• Benchmarking Embedding Aggregation Methods in Computational Pathology: A Clinical Data Perspective [32.93871326428446]
  Recent advances in artificial intelligence (AI) are revolutionizing medical imaging and computational pathology. A constant challenge in the analysis of digital Whole Slide Images (WSIs) is the problem of aggregating tens of thousands of tile-level image embeddings to a slide-level representation. This study conducts a benchmarking analysis of ten slide-level aggregation techniques across nine clinically relevant tasks.
  arXiv  Detail & Related papers  (2024-07-10T17:00:57Z)
• Kernel Density Estimation for Multiclass Quantification [52.419589623702336]
  Quantification is the supervised machine learning task concerned with obtaining accurate predictors of class prevalence. The distribution-matching (DM) approaches represent one of the most important families among the quantification methods that have been proposed in the literature so far. We propose a new representation mechanism based on multivariate densities that we model via kernel density estimation (KDE)
  arXiv  Detail & Related papers  (2023-12-31T13:19:27Z)
• MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation [58.93221876843639]
  This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion. It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space. It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
  arXiv  Detail & Related papers  (2023-10-04T01:36:30Z)
• Using Large Language Models to Automate Category and Trend Analysis of Scientific Articles: An Application in Ophthalmology [4.455826633717872]
  We present an automated method for article classification, leveraging the power of Large Language Models (LLM) The model achieved mean accuracy of 0.86 and mean F1 of 0.85 based on the RenD dataset. The extendibility of the model to other scientific fields broadens its impact in facilitating research and trend analysis across diverse disciplines.
  arXiv  Detail & Related papers  (2023-08-31T12:45:53Z)
• Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functions [30.305418761024143]
  Real-world drug discovery tasks are often characterized by a scarcity of labeled data and a significant range of data. We present a principled way to encode explicit prior knowledge of the data-generating process into a prior distribution. We demonstrate that using integrate Q-SAVI to contextualize prior knowledgelike chemical space into the modeling process affords substantial accuracy and calibration.
  arXiv  Detail & Related papers  (2023-07-14T05:01:10Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• An efficient representation of chronological events in medical texts [9.118144540451514]
  We proposed a systematic methodology for learning from chronological events available in clinical notes. The proposed methodological it path signature framework creates a non-parametric hierarchical representation of sequential events of any type. The methodology was developed and externally validated using the largest in the UK secondary care mental health EHR data.
  arXiv  Detail & Related papers  (2020-10-16T14:54:29Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.