Overview of the BioLaySumm 2024 Shared Task on the Lay Summarization of Biomedical Research Articles

• URL: http://arxiv.org/abs/2408.08566v1
• Date: Fri, 16 Aug 2024 07:00:08 GMT
• Title: Overview of the BioLaySumm 2024 Shared Task on the Lay Summarization of Biomedical Research Articles
• Authors: Tomas Goldsack, Carolina Scarton, Matthew Shardlow, Chenghua Lin,
• Abstract summary: This paper presents the setup and results of the second edition of the BioLaySumm shared task on the Lay Summarisation of Biomedical Research Articles. We aim to build on the first edition's success by further increasing research interest in this important task and encouraging participants to explore novel approaches. Overall, our results show that a broad range of innovative approaches were adopted by task participants, with a predictable shift towards the use of Large Language Models (LLMs)
• Score: 21.856049605149646
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents the setup and results of the second edition of the BioLaySumm shared task on the Lay Summarisation of Biomedical Research Articles, hosted at the BioNLP Workshop at ACL 2024. In this task edition, we aim to build on the first edition's success by further increasing research interest in this important task and encouraging participants to explore novel approaches that will help advance the state-of-the-art. Encouragingly, we found research interest in the task to be high, with this edition of the task attracting a total of 53 participating teams, a significant increase in engagement from the previous edition. Overall, our results show that a broad range of innovative approaches were adopted by task participants, with a predictable shift towards the use of Large Language Models (LLMs).

Related papers

• SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories [55.161075901665946]
  Super aims to capture the realistic challenges faced by researchers working with Machine Learning (ML) and Natural Language Processing (NLP) research repositories. Our benchmark comprises three distinct problem sets: 45 end-to-end problems with annotated expert solutions, 152 sub problems derived from the expert set that focus on specific challenges, and 602 automatically generated problems for larger-scale development. We show that state-of-the-art approaches struggle to solve these problems with the best model (GPT-4o) solving only 16.3% of the end-to-end set, and 46.1% of the scenarios.
  arXiv  Detail & Related papers  (2024-09-11T17:37:48Z)
• Intent Detection and Entity Extraction from BioMedical Literature [14.52164637112797]
  Large-language models (LLMs) motivated by endeavours to attain generalized intelligence, their efficacy in replacing task and domain-specific natural language understanding approaches remains questionable. We show that Supervised Fine Tuned approaches are still relevant and more effective than general-purpose LLMs.
  arXiv  Detail & Related papers  (2024-04-04T17:09:52Z)
• An Evaluation of Large Language Models in Bioinformatics Research [52.100233156012756]
  We study the performance of large language models (LLMs) on a wide spectrum of crucial bioinformatics tasks. These tasks include the identification of potential coding regions, extraction of named entities for genes and proteins, detection of antimicrobial and anti-cancer peptides, molecular optimization, and resolution of educational bioinformatics problems. Our findings indicate that, given appropriate prompts, LLMs like GPT variants can successfully handle most of these tasks.
  arXiv  Detail & Related papers  (2024-02-21T11:27:31Z)
• ProBio: A Protocol-guided Multimodal Dataset for Molecular Biology Lab [67.24684071577211]
  The challenge of replicating research results has posed a significant impediment to the field of molecular biology. We first curate a comprehensive multimodal dataset, named ProBio, as an initial step towards this objective. Next, we devise two challenging benchmarks, transparent solution tracking and multimodal action recognition, to emphasize the unique characteristics and difficulties associated with activity understanding in BioLab settings.
  arXiv  Detail & Related papers  (2023-11-01T14:44:01Z)
• Overview of the BioLaySumm 2023 Shared Task on Lay Summarization of Biomedical Research Articles [47.04555835353173]
  This paper presents the results of the shared task on Lay Summarisation of Biomedical Research Articles (BioLaySumm) hosted at the BioNLP Workshop at ACL 2023. The goal of this shared task is to develop abstractive summarisation models capable of generating "lay summaries" In addition to overall results, we report on the setup and insights from the BioLaySumm shared task, which attracted a total of 20 participating teams across both subtasks.
  arXiv  Detail & Related papers  (2023-09-29T15:43:42Z)
• Overview of BioASQ 2023: The eleventh BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering [0.1759008116536278]
  BioASQ is a series of challenges promoting advances in large-scale biomedical semantic indexing and question answering. This year, BioASQ consisted of new editions of the two established tasks b and Synergy, and a new task (MedProcNER) on semantic annotation of clinical content in Spanish with medical procedures. In this edition of BioASQ, 28 competing teams submitted the results of more than 150 distinct systems in total for the three different shared tasks of the challenge.
  arXiv  Detail & Related papers  (2023-07-11T09:20:33Z)
• Parsing Objects at a Finer Granularity: A Survey [54.72819146263311]
  Fine-grained visual parsing is important in many real-world applications, e.g., agriculture, remote sensing, and space technologies. Predominant research efforts tackle these fine-grained sub-tasks following different paradigms. We conduct an in-depth study of the advanced work from a new perspective of learning the part relationship.
  arXiv  Detail & Related papers  (2022-12-28T04:20:10Z)
• LED down the rabbit hole: exploring the potential of global attention for biomedical multi-document summarisation [59.307534363825816]
  We adapt PRIMERA to the biomedical domain by placing global attention on important biomedical entities. We analyse the outputs of the 23 resulting models, and report patterns in the results related to the presence of additional global attention.
  arXiv  Detail & Related papers  (2022-09-19T01:13:42Z)
• Overview and Insights from the SciVer Shared Task on Scientific Claim Verification [5.78530472626281]
  We present an overview of the SciVer shared task, presented at the 2nd Scholarly Document Processing (SDP) workshop at NAACL 2021. 11 teams made a total of 14 submissions to the shared task leaderboard, leading to an improvement of more than +23 F1 on the primary task evaluation metric.
  arXiv  Detail & Related papers  (2021-07-17T05:47:57Z)
• Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering [0.293168019422713]
  The BioASQ challenge aims to advance the state-of-the-art in large-scale biomedical semantic indexing and question answering. This paper presents an overview of the ninth edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2021. In total, 42 teams with more than 170 systems were registered to participate in the four tasks of the challenge.
  arXiv  Detail & Related papers  (2021-06-28T10:03:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.