Automating Knowledge Discovery from Scientific Literature via LLMs: A Dual-Agent Approach with Progressive Ontology Prompting

• URL: http://arxiv.org/abs/2409.00054v1
• Date: Tue, 20 Aug 2024 16:42:23 GMT
• Title: Automating Knowledge Discovery from Scientific Literature via LLMs: A Dual-Agent Approach with Progressive Ontology Prompting
• Authors: Yuting Hu, Dancheng Liu, Qingyun Wang, Charles Yu, Heng Ji, Jinjun Xiong,
• Abstract summary: We introduce a novel framework based on large language models (LLMs) that combines a progressive prompting algorithm with a dual-agent system, named LLM-Duo. Our method identifies 2,421 interventions from 64,177 research articles in the speech-language therapy domain.
• Score: 59.97247234955861
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To address the challenge of automating knowledge discovery from a vast volume of literature, in this paper, we introduce a novel framework based on large language models (LLMs) that combines a progressive ontology prompting (POP) algorithm with a dual-agent system, named LLM-Duo, designed to enhance the automation of knowledge extraction from scientific articles. The POP algorithm utilizes a prioritized breadth-first search (BFS) across a predefined ontology to generate structured prompt templates and action orders, thereby guiding LLMs to discover knowledge in an automatic manner. Additionally, our LLM-Duo employs two specialized LLM agents: an explorer and an evaluator. These two agents work collaboratively and adversarially to enhance the reliability of the discovery and annotation processes. Experiments demonstrate that our method outperforms advanced baselines, enabling more accurate and complete annotations. To validate the effectiveness of our method in real-world scenarios, we employ our method in a case study of speech-language intervention discovery. Our method identifies 2,421 interventions from 64,177 research articles in the speech-language therapy domain. We curate these findings into a publicly accessible intervention knowledge base that holds significant potential to benefit the speech-language therapy community.

Related papers

• Enriching Ontologies with Disjointness Axioms using Large Language Models [5.355177558868206]
  Large Models (LLMs) offer consistency by identifying and asserting class disjointness axioms. Our approach aims at leveraging the implicit knowledge embedded in LLMs to elicit knowledge for classifying ontological disjointness. Our findings suggest that LLMs, when guided by effective prompt strategies, can reliably identify disjointness relationships.
  arXiv  Detail & Related papers  (2024-10-04T09:00:06Z)
• Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature [48.572336666741194]
  We present Knowledge Navigator, a system designed to enhance exploratory search abilities. It organizes retrieved documents into a navigable, two-level hierarchy of named and descriptive scientific topics and subtopics.
  arXiv  Detail & Related papers  (2024-08-28T14:48:37Z)
• A Quick, trustworthy spectral knowledge Q&A system leveraging retrieval-augmented generation on LLM [0.0]
  Large Language Model (LLM) has demonstrated significant success in a range of natural language processing (NLP) tasks within general domain. We introduce the Spectral Detection and Analysis Based Paper (SDAAP) dataset, which is the first open-source textual knowledge dataset for spectral analysis and detection. We also designed an automated Q&A framework based on the SDAAP dataset, which can retrieve relevant knowledge and generate high-quality responses.
  arXiv  Detail & Related papers  (2024-08-21T12:09:37Z)
• Efficient Prompting Methods for Large Language Models: A Survey [50.171011917404485]
  Prompting has become a mainstream paradigm for adapting large language models (LLMs) to specific natural language processing tasks. This approach brings the additional computational burden of model inference and human effort to guide and control the behavior of LLMs. We present the basic concepts of prompting, review the advances for efficient prompting, and highlight future research directions.
  arXiv  Detail & Related papers  (2024-04-01T12:19:08Z)
• FAC$^2$E: Better Understanding Large Language Model Capabilities by Dissociating Language and Cognition [56.76951887823882]
  Large language models (LLMs) are primarily evaluated by overall performance on various text understanding and generation tasks. We present FAC$2$E, a framework for Fine-grAined and Cognition-grounded LLMs' Capability Evaluation.
  arXiv  Detail & Related papers  (2024-02-29T21:05:37Z)
• Automating Knowledge Acquisition for Content-Centric Cognitive Agents Using LLMs [0.0]
  The paper describes a system that uses large language model (LLM) technology to support the automatic learning of new entries in an intelligent agent's semantic lexicon. The process is bootstrapped by an existing non-toy lexicon and a natural language generator that converts formal, ontologically-grounded representations of meaning into natural language sentences.
  arXiv  Detail & Related papers  (2023-12-27T02:31:51Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)
• LLMs4OL: Large Language Models for Ontology Learning [0.0]
  We propose the LLMs4OL approach, which utilizes Large Language Models (LLMs) for Ontology Learning (OL) LLMs have shown significant advancements in natural language processing, demonstrating their ability to capture complex language patterns in different knowledge domains. The evaluations encompass diverse genres of ontological knowledge, including lexicosemantic knowledge in WordNet, geographical knowledge in GeoNames, and medical knowledge in UMLS.
  arXiv  Detail & Related papers  (2023-07-31T13:27:21Z)
• Red Teaming Language Model Detectors with Language Models [114.36392560711022]
  Large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. Recent works have proposed algorithms to detect LLM-generated text and protect LLMs. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation.
  arXiv  Detail & Related papers  (2023-05-31T10:08:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.