EHRAgent: Code Empowers Large Language Models for Few-shot Complex
Tabular Reasoning on Electronic Health Records
- URL: http://arxiv.org/abs/2401.07128v2
- Date: Mon, 19 Feb 2024 21:47:41 GMT
- Title: EHRAgent: Code Empowers Large Language Models for Few-shot Complex
Tabular Reasoning on Electronic Health Records
- Authors: Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda
Zhu, Joyce Ho, Carl Yang, May D. Wang
- Abstract summary: Large language models (LLMs) have demonstrated exceptional capabilities in planning and tool utilization.
We propose EHRAgent, an LLM agent empowered with a code interface, to autonomously generate and execute code for multi-tabular reasoning.
- Score: 48.9788858020623
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large language models (LLMs) have demonstrated exceptional capabilities in
planning and tool utilization as autonomous agents, but few have been developed
for medical problem-solving. We propose EHRAgent, an LLM agent empowered with a
code interface, to autonomously generate and execute code for multi-tabular
reasoning within electronic health records (EHRs). First, we formulate an EHR
question-answering task into a tool-use planning process, efficiently
decomposing a complicated task into a sequence of manageable actions. By
integrating interactive coding and execution feedback, EHRAgent learns from
error messages and improves the originally generated code through iterations.
Furthermore, we enhance the LLM agent by incorporating long-term memory, which
allows EHRAgent to effectively select and build upon the most relevant
successful cases from past experiences. Experiments on three real-world
multi-tabular EHR datasets show that EHRAgent outperforms the strongest
baseline by up to 29.6% in success rate. EHRAgent leverages the emerging
few-shot learning capabilities of LLMs, enabling autonomous code generation and
execution to tackle complex clinical tasks with minimal demonstrations.
Related papers
- MMedAgent: Learning to Use Medical Tools with Multi-modal Agent [26.315786330786676]
This paper introduces the first agent explicitly designed for the medical field, named textbfMulti-modal textbfMedical textbfAgent (MMedAgent)
We curate an instruction-tuning dataset comprising six medical tools solving seven tasks, enabling the agent to choose the most suitable tools for a given task.
MMedAgent achieves superior performance across a variety of medical tasks compared to state-of-the-art open-source methods and even the closed-source model, GPT-4o.
arXiv Detail & Related papers (2024-07-02T17:58:23Z) - CAAP: Context-Aware Action Planning Prompting to Solve Computer Tasks with Front-End UI Only [21.054681757006385]
Large Language Models (LLMs) with advanced reasoning capabilities have set the stage for agents to undertake more complex and previously unseen tasks.
We propose an agent that functions solely on the basis of screenshots for recognizing environments.
We achieve a success rate of 94.4% on 67types of MiniWoB++ problems, utilizing only 1.48demonstrations per problem type.
arXiv Detail & Related papers (2024-06-11T05:21:20Z) - Exploring LLM Multi-Agents for ICD Coding [15.730751450511333]
We present a novel multi-agent method for ICD coding, which mimics the real-world coding process with five agents.
We show that our proposed multi-agent coding framework substantially improves performance on both common and rare codes.
Our method also matches the state-of-the-art ICD coding methods that require pre-training or fine-tuning.
arXiv Detail & Related papers (2024-04-01T15:17:39Z) - Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning [56.82041895921434]
Open-source pre-trained Large Language Models (LLMs) exhibit strong language understanding and generation capabilities.
When used as agents for dealing with complex problems in the real world, their performance is far inferior to large commercial models such as ChatGPT and GPT-4.
arXiv Detail & Related papers (2024-03-29T03:48:12Z) - Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models [56.00992369295851]
Open-sourced Large Language Models (LLMs) have achieved great success in various NLP tasks, however, they are still far inferior to API-based models when acting as agents.
This paper delivers three key observations: (1) the current agent training corpus is entangled with both formats following and agent reasoning, which significantly shifts from the distribution of its pre-training data; (2) LLMs exhibit different learning speeds on the capabilities required by agent tasks; and (3) current approaches have side-effects when improving agent abilities by introducing hallucinations.
We propose Agent-FLAN to effectively Fine-tune LANguage models for Agents.
arXiv Detail & Related papers (2024-03-19T16:26:10Z) - DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning [56.887047551101574]
We present DS-Agent, a novel framework that harnesses large language models (LLMs) agent and case-based reasoning (CBR)
In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle.
In the deployment stage, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm, significantly reducing the demand on foundational capabilities of LLMs.
arXiv Detail & Related papers (2024-02-27T12:26:07Z) - Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with
Agent Team Optimization [59.39113350538332]
Large language model (LLM) agents have been shown effective on a wide range of tasks, and by ensembling multiple LLM agents, their performances could be further improved.
Existing approaches employ a fixed set of agents to interact with each other in a static architecture.
We build a framework named Dynamic LLM-Agent Network ($textbfDyLAN$) for LLM-agent collaboration on complicated tasks like reasoning and code generation.
arXiv Detail & Related papers (2023-10-03T16:05:48Z) - Recommender AI Agent: Integrating Large Language Models for Interactive
Recommendations [53.76682562935373]
We introduce an efficient framework called textbfInteRecAgent, which employs LLMs as the brain and recommender models as tools.
InteRecAgent achieves satisfying performance as a conversational recommender system, outperforming general-purpose LLMs.
arXiv Detail & Related papers (2023-08-31T07:36:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.