LLMs4Synthesis: Leveraging Large Language Models for Scientific Synthesis

• URL: http://arxiv.org/abs/2409.18812v1
• Date: Fri, 27 Sep 2024 15:04:39 GMT
• Title: LLMs4Synthesis: Leveraging Large Language Models for Scientific Synthesis
• Authors: Hamed Babaei Giglou, Jennifer D'Souza, Sören Auer,
• Abstract summary: This paper introduces the LLMs4Synthesis framework, designed to enhance the capabilities of Large Language Models (LLMs) in generating high-quality scientific syntheses. It addresses the need for rapid, coherent, and contextually rich integration of scientific insights, leveraging both open-source and proprietary LLMs.
• Score: 0.16385815610837165
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In response to the growing complexity and volume of scientific literature, this paper introduces the LLMs4Synthesis framework, designed to enhance the capabilities of Large Language Models (LLMs) in generating high-quality scientific syntheses. This framework addresses the need for rapid, coherent, and contextually rich integration of scientific insights, leveraging both open-source and proprietary LLMs. It also examines the effectiveness of LLMs in evaluating the integrity and reliability of these syntheses, alleviating inadequacies in current quantitative metrics. Our study contributes to this field by developing a novel methodology for processing scientific papers, defining new synthesis types, and establishing nine detailed quality criteria for evaluating syntheses. The integration of LLMs with reinforcement learning and AI feedback is proposed to optimize synthesis quality, ensuring alignment with established criteria. The LLMs4Synthesis framework and its components are made available, promising to enhance both the generation and evaluation processes in scientific research synthesis.

Related papers

• Leveraging Generative AI to Enhance Synthea Module Development [0.044998333629984864]
  This paper explores the use of large language models (LLMs) to assist in the development of new disease modules for Synthea, an open-source synthetic health data generator.<n> incorporating LLMs into the module development process has the potential to reduce development time, reduce required expertise, expand model diversity, and improve the overall quality of synthetic patient data.
  arXiv  Detail & Related papers  (2025-07-18T17:44:35Z)
• ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data [53.78763789036172]
  We present ChemActor, a fully fine-tuned large language model (LLM) as a chemical executor to convert between unstructured experimental procedures and structured action sequences.<n>This framework integrates a data selection module that selects data based on distribution divergence, with a general-purpose LLM, to generate machine-executable actions from a single molecule input.<n>Experiments on reaction-to-description (R2D) and description-to-action (D2A) tasks demonstrate that ChemActor achieves state-of-the-art performance, outperforming the baseline model by 10%.
  arXiv  Detail & Related papers  (2025-06-30T05:11:19Z)
• Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning [80.27561080938747]
  We propose a systematic framework, CANOE, to improve the faithfulness of large language models (LLMs) in both short-form and long-form generation tasks without human annotations.<n>Also, we propose Dual-GRPO, a rule-based reinforcement learning method that includes three tailored rule-based rewards derived from synthesized short-form QA data.<n> Experimental results show that CANOE greatly improves the faithfulness of LLMs across 11 different downstream tasks, even outperforming the most advanced LLMs.
  arXiv  Detail & Related papers  (2025-05-22T10:10:07Z)
• Reshaping MOFs text mining with a dynamic multi-agents framework of large language model [5.150905688058796]
  We present MOFh6, a large language model (LLM)-based multi-agent system designed to extract, structure, and apply synthesis knowledge.<n>MoFh6 achieves 99% accuracy in synthesis data parsing and resolves 94.1% of complex co-reference abbreviations.<n>It processes a single full-text document in 9.6 seconds and localizes structured synthesis descriptions within 36 seconds, with the cost per 100 papers reduced to USD 4.24, a 76% saving over existing systems.
  arXiv  Detail & Related papers  (2025-04-26T09:55:04Z)
• A Survey on Post-training of Large Language Models [185.51013463503946]
  Large Language Models (LLMs) have fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific exploration. These challenges necessitate advanced post-training language models (PoLMs) to address shortcomings, such as restricted reasoning capacities, ethical uncertainties, and suboptimal domain-specific performance. This paper presents the first comprehensive survey of PoLMs, systematically tracing their evolution across five core paradigms.
  arXiv  Detail & Related papers  (2025-03-08T05:41:42Z)
• Towards Fully-Automated Materials Discovery via Large-Scale Synthesis Dataset and Expert-Level LLM-as-a-Judge [6.500470477634259]
  Our work aims to support the materials science community by providing a practical, data-driven resource. We have curated a comprehensive dataset of 17K expert-verified synthesis recipes from open-access literature. AlchemicalBench offers an end-to-end framework that supports research in large language models applied to synthesis prediction.
  arXiv  Detail & Related papers  (2025-02-23T06:16:23Z)
• A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences [1.8308043661908204]
  This paper reviews the state-of-the-art applications of large language models (LLMs) in the biomedical domain. LLMs demonstrate remarkable potential, but significant challenges remain, including issues related to hallucinations, contextual understanding, and the ability to generalize. We aim to improve access to medical literature and facilitate meaningful discoveries in healthcare.
  arXiv  Detail & Related papers  (2024-12-04T18:26:13Z)
• Self-Boosting Large Language Models with Synthetic Preference Data [97.94185115047999]
  We introduce SynPO, a self-boosting paradigm that leverages synthetic preference data for model alignment. After four SynPO iterations, Llama3-8B and Mistral-7B show significant enhancements in instruction-following abilities. SynPO improves the general performance of LLMs on various tasks, validated by a 3.2 to 5.0 average score increase on the well-recognized Open LLM leaderboard.
  arXiv  Detail & Related papers  (2024-10-09T14:57:31Z)
• Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models [54.51932175059004]
  We introduce a scalable method for generating synthetic instructions to enhance the code generation capability of Large Language Models. The proposed algorithm, Genetic-Instruct, mimics evolutionary processes, utilizing self-instruction to create numerous synthetic samples from a limited number of seeds.
  arXiv  Detail & Related papers  (2024-07-29T20:42:59Z)
• Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
  Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML) This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature. The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
  arXiv  Detail & Related papers  (2024-07-17T20:01:21Z)
• Large Language Models as Evaluators for Scientific Synthesis [0.16385815610837165]
  We use a dataset of 100 research questions and their syntheses made by GPT-4 from abstracts of five related papers, checked against human quality ratings. Preliminary results show that LLMs can offer logical explanations that somewhat match the quality ratings, yet a deeper statistical analysis shows a weak correlation between LLM and human ratings.
  arXiv  Detail & Related papers  (2024-07-03T10:21:27Z)
• A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery [68.48094108571432]
  Large language models (LLMs) have revolutionized the way text and other modalities of data are handled. We aim to provide a more holistic view of the research landscape by unveiling cross-field and cross-modal connections between scientific LLMs.
  arXiv  Detail & Related papers  (2024-06-16T08:03:24Z)
• ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models [56.08917291606421]
  ResearchAgent is an AI-based system for ideation and operationalization of novel work. ResearchAgent automatically defines novel problems, proposes methods and designs experiments, while iteratively refining them. We experimentally validate our ResearchAgent on scientific publications across multiple disciplines.
  arXiv  Detail & Related papers  (2024-04-11T13:36:29Z)
• Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning [0.9110413356918055]
  This research pioneers the use of fine-tuned Large Language Models (LLMs) to automate Systematic Literature Reviews ( SLRs) Our study employed the latest fine-tuning methodologies together with open-sourced LLMs, and demonstrated a practical and efficient approach to automating the final execution stages of an SLR process. The results maintained high fidelity in factual accuracy in LLM responses, and were validated through the replication of an existing PRISMA-conforming SLR.
  arXiv  Detail & Related papers  (2024-04-08T00:08:29Z)
• Guiding Enumerative Program Synthesis with Large Language Models [15.500250058226474]
  In this paper, we evaluate the abilities of Large Language Models to solve formal synthesis benchmarks. When one-shot synthesis fails, we propose a novel enumerative synthesis algorithm. We find that GPT-3.5 as a stand-alone tool for formal synthesis is easily outperformed by state-of-the-art formal synthesis algorithms.
  arXiv  Detail & Related papers  (2024-03-06T19:13:53Z)
• SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis [26.111514038691837]
  SciAssess is a benchmark for the comprehensive evaluation of Large Language Models (LLMs) in scientific literature analysis. It aims to thoroughly assess the efficacy of LLMs by evaluating their capabilities in Memorization (L1), memorization (L2), and Analysis & Reasoning (L3) It encompasses a variety of tasks drawn from diverse scientific fields, including biology, chemistry, material, and medicine.
  arXiv  Detail & Related papers  (2024-03-04T12:19:28Z)
• Boosting Large Language Model for Speech Synthesis: An Empirical Study [86.89548753080432]
  Large language models (LLMs) have made significant advancements in natural language processing and are concurrently extending the language ability to other modalities, such as speech and vision. We conduct a comprehensive empirical exploration of boosting LLMs with the ability to generate speech, by combining pre-trained LLM LLaMA/OPT and text-to-speech synthesis model VALL-E. We compare three integration methods between LLMs and speech models, including directly fine-tuned LLMs, superposed layers of LLMs and VALL-E, and coupled LLMs and VALL-E using LLMs as a powerful text encoder
  arXiv  Detail & Related papers  (2023-12-30T14:20:04Z)
• Summarization is (Almost) Dead [49.360752383801305]
  We develop new datasets and conduct human evaluation experiments to evaluate the zero-shot generation capability of large language models (LLMs) Our findings indicate a clear preference among human evaluators for LLM-generated summaries over human-written summaries and summaries generated by fine-tuned models.
  arXiv  Detail & Related papers  (2023-09-18T08:13:01Z)
• ULSA: Unified Language of Synthesis Actions for Representation of Synthesis Protocols [2.436060325115753]
  We propose the first Unified Language of Synthesis Actions (ULSA) for describing synthesis procedures. We created a dataset of 3,040 synthesis procedures annotated by domain experts according to the proposed ULSA scheme.
  arXiv  Detail & Related papers  (2022-01-23T17:44:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.