Large Language Models and Prompt Engineering for Biomedical Query Focused Multi-Document Summarisation

• URL: http://arxiv.org/abs/2311.05169v1
• Date: Thu, 9 Nov 2023 06:45:04 GMT
• Title: Large Language Models and Prompt Engineering for Biomedical Query Focused Multi-Document Summarisation
• Authors: Diego Moll\'a
• Abstract summary: This paper reports on the use of prompt engineering and GPT-3.5 for biomedical query-focused multi-document summarisation. Using GPT-3.5 and appropriate prompts, our system top ROUGE-F1 results in the task of obtaining short-paragraph-sized answers to biomedical questions.
• Score: 0.565658124285176
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper reports on the use of prompt engineering and GPT-3.5 for biomedical query-focused multi-document summarisation. Using GPT-3.5 and appropriate prompts, our system achieves top ROUGE-F1 results in the task of obtaining short-paragraph-sized answers to biomedical questions in the 2023 BioASQ Challenge (BioASQ 11b). This paper confirms what has been observed in other domains: 1) Prompts that incorporated few-shot samples generally improved on their counterpart zero-shot variants; 2) The largest improvement was achieved by retrieval augmented generation. The fact that these prompts allow our top runs to rank within the top two runs of BioASQ 11b demonstrate the power of using adequate prompts for Large Language Models in general, and GPT-3.5 in particular, for query-focused summarisation.

Related papers

• Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions [1.0742675209112622]
  We propose a two-level information retrieval and question-answering system based on pre-trained large language models (LLM) We construct prompts with in-context few-shot examples and utilize post-processing techniques like resampling and malformed response detection. Our best-performing system achieved 0.14 MAP score on document retrieval, 0.05 MAP score on snippet retrieval, 0.96 F1 score for yes/no questions, 0.38 MRR score for factoid questions and 0.50 F1 score for list questions in Task 12b.
  arXiv  Detail & Related papers  (2024-07-09T11:48:49Z)
• WisPerMed at BioLaySumm: Adapting Autoregressive Large Language Models for Lay Summarization of Scientific Articles [0.41716369948557463]
  This paper details the efforts of the WisPerMed team in the BioLaySumm2024 Shared Task on automatic lay summarization. Large language models (LLMs), specifically the BioMistral and Llama3 models, were fine-tuned and employed to create lay summaries. Experiments demonstrated that fine-tuning generally led to the best performance across most evaluated metrics.
  arXiv  Detail & Related papers  (2024-05-20T10:54:47Z)
• BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers [48.21255861863282]
  BMRetriever is a series of dense retrievers for enhancing biomedical retrieval. BMRetriever exhibits strong parameter efficiency, with the 410M variant outperforming baselines up to 11.7 times larger.
  arXiv  Detail & Related papers  (2024-04-29T05:40:08Z)
• BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning [77.90250740041411]
  This paper introduces BioT5+, an extension of the BioT5 framework, tailored to enhance biological research and drug discovery. BioT5+ incorporates several novel features: integration of IUPAC names for molecular understanding, inclusion of extensive bio-text and molecule data from sources like bioRxiv and PubChem, the multi-task instruction tuning for generality across tasks, and a numerical tokenization technique for improved processing of numerical data.
  arXiv  Detail & Related papers  (2024-02-27T12:43:09Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts [16.05119302860606]
  We investigate the ability of state-of-the-art large language models (LLMs) on the task of biomedical abstract simplification. The methods applied include domain fine-tuning and prompt-based learning (PBL) We used a range of automatic evaluation metrics, including BLEU, ROUGE, SARI, and BERTscore, and also conducted human evaluations.
  arXiv  Detail & Related papers  (2023-09-22T22:47:32Z)
• BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks [68.39821375903591]
  Generalist AI holds the potential to address limitations due to its versatility in interpreting different data types. Here, we propose BiomedGPT, the first open-source and lightweight vision-language foundation model.
  arXiv  Detail & Related papers  (2023-05-26T17:14:43Z)
• Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success) [36.646495151276326]
  GPT-3 is able to produce high quality summaries of general domain news articles in few- and zero-shot settings. We enlist domain experts (individuals with medical training) to evaluate summaries of biomedical articles generated by GPT-3, given zero supervision.
  arXiv  Detail & Related papers  (2023-05-10T16:40:37Z)
• PMC-LLaMA: Towards Building Open-source Language Models for Medicine [62.39105735933138]
  Large Language Models (LLMs) have showcased remarkable capabilities in natural language understanding. LLMs struggle in domains that require precision, such as medical applications, due to their lack of domain-specific knowledge. We describe the procedure for building a powerful, open-source language model specifically designed for medicine applications, termed as PMC-LLaMA.
  arXiv  Detail & Related papers  (2023-04-27T18:29:05Z)
• BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining [140.61707108174247]
  We propose BioGPT, a domain-specific generative Transformer language model pre-trained on large scale biomedical literature. We get 44.98%, 38.42% and 40.76% F1 score on BC5CDR, KD-DTI and DDI end-to-end relation extraction tasks respectively, and 78.2% accuracy on PubMedQA.
  arXiv  Detail & Related papers  (2022-10-19T07:17:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.