Good Data, Large Data, or No Data? Comparing Three Approaches in Developing Research Aspect Classifiers for Biomedical Papers

• URL: http://arxiv.org/abs/2306.04820v1
• Date: Wed, 7 Jun 2023 22:56:53 GMT
• Title: Good Data, Large Data, or No Data? Comparing Three Approaches in Developing Research Aspect Classifiers for Biomedical Papers
• Authors: Shreya Chandrasekhar, Chieh-Yang Huang, Ting-Hao 'Kenneth' Huang
• Abstract summary: We investigate the impact of different datasets on model performance for the crowd-annotated CODA-19 research aspect classification task. Our results indicate that using the PubMed 200K RCT dataset does not improve performance for the CODA-19 task.
• Score: 19.1408856831043
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid growth of scientific publications, particularly during the COVID-19 pandemic, emphasizes the need for tools to help researchers efficiently comprehend the latest advancements. One essential part of understanding scientific literature is research aspect classification, which categorizes sentences in abstracts to Background, Purpose, Method, and Finding. In this study, we investigate the impact of different datasets on model performance for the crowd-annotated CODA-19 research aspect classification task. Specifically, we explore the potential benefits of using the large, automatically curated PubMed 200K RCT dataset and evaluate the effectiveness of large language models (LLMs), such as LLaMA, GPT-3, ChatGPT, and GPT-4. Our results indicate that using the PubMed 200K RCT dataset does not improve performance for the CODA-19 task. We also observe that while GPT-4 performs well, it does not outperform the SciBERT model fine-tuned on the CODA-19 dataset, emphasizing the importance of a dedicated and task-aligned datasets dataset for the target task. Our code is available at https://github.com/Crowd-AI-Lab/CODA-19-exp.

Related papers

• SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature [80.49349719239584]
  We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following demonstrations for 54 tasks. SciRIFF is the first dataset focused on extracting and synthesizing information from research literature across a wide range of scientific fields.
  arXiv  Detail & Related papers  (2024-06-10T21:22:08Z)
• Dataset Mention Extraction in Scientific Articles Using Bi-LSTM-CRF Model [0.0]
  We show that citing datasets is not a common or standard practice in spite of recent efforts by data repositories and funding agencies. A potential solution to this problem is to automatically extract dataset mentions from scientific articles. In this work, we propose to achieve such extraction by using a neural network based on a Bi-LSTM-CRF architecture.
  arXiv  Detail & Related papers  (2024-05-21T18:12:37Z)
• Enriched BERT Embeddings for Scholarly Publication Classification [0.13654846342364302]
  The NSLP 2024 FoRC Task I addresses this challenge organized as a competition. The goal is to develop a classifier capable of predicting one of 123 predefined classes from the Open Research Knowledge Graph (ORKG) taxonomy of research fields for a given article.
  arXiv  Detail & Related papers  (2024-05-07T09:05:20Z)
• Robust COVID-19 Detection in CT Images with CLIP [21.809469794865887]
  Deep learning models face challenges in medical imaging, particularly for COVID-19 detection. We introduce the first lightweight detector designed to overcome these obstacles, leveraging a frozen CLIP image encoder and a trainable multilayer perception (MLP) We integrate a teacher-student framework to capitalize on the vast amounts of unlabeled data, enabling our model to achieve superior performance despite the inherent data limitations.
  arXiv  Detail & Related papers  (2024-03-13T20:26:50Z)
• An Information Retrieval and Extraction Tool for Covid-19 Related Papers [0.0]
  The main focus of this paper is to provide researchers with a better search tool for COVID-19 related papers. Our tool has shown the potential to assist researchers by automating a topic-based search of CORD-19 papers.
  arXiv  Detail & Related papers  (2024-01-20T01:34:50Z)
• Evaluating the Fairness of the MIMIC-IV Dataset and a Baseline Algorithm: Application to the ICU Length of Stay Prediction [65.268245109828]
  This paper uses the MIMIC-IV dataset to examine the fairness and bias in an XGBoost binary classification model predicting the ICU length of stay. The research reveals class imbalances in the dataset across demographic attributes and employs data preprocessing and feature extraction. The paper concludes with recommendations for fairness-aware machine learning techniques for mitigating biases and the need for collaborative efforts among healthcare professionals and data scientists.
  arXiv  Detail & Related papers  (2023-12-31T16:01:48Z)
• Replication: Contrastive Learning and Data Augmentation in Traffic Classification Using a Flowpic Input Representation [47.95762911696397]
  We reproduce [16] on the same datasets and replicate its most salient aspect (the importance of data augmentation) on three additional public datasets. While we confirm most of the original results, we also found a 20% accuracy drop on some of the investigated scenarios due to a data shift in the original dataset.
  arXiv  Detail & Related papers  (2023-09-18T12:55:09Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• Evaluating the effect of data augmentation and BALD heuristics on distillation of Semantic-KITTI dataset [63.20765930558542]
  Active Learning has remained relatively unexplored for LiDAR perception tasks in autonomous driving datasets. We evaluate Bayesian active learning methods applied to the task of dataset distillation or core subset selection. We also study the effect of application of data augmentation within Bayesian AL based dataset distillation.
  arXiv  Detail & Related papers  (2023-02-21T13:56:47Z)
• Efficient Analysis of COVID-19 Clinical Data using Machine Learning Models [0.0]
  Huge volumes of data and case studies have been made available, providing researchers with a unique opportunity to find trends. Applying machine learning based algorithms to this big data is a natural approach to take to this aim. We show that with the efficient feature selection algorithm, we can achieve a prediction accuracy of more than 90% in most cases.
  arXiv  Detail & Related papers  (2021-10-18T20:06:01Z)
• Deep learning-based COVID-19 pneumonia classification using chest CT images: model generalizability [54.86482395312936]
  Deep learning (DL) classification models were trained to identify COVID-19-positive patients on 3D computed tomography (CT) datasets from different countries. We trained nine identical DL-based classification models by using combinations of the datasets with a 72% train, 8% validation, and 20% test data split. The models trained on multiple datasets and evaluated on a test set from one of the datasets used for training performed better.
  arXiv  Detail & Related papers  (2021-02-18T21:14:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.