TEXT2DB: Integration-Aware Information Extraction with Large Language Model Agents

• URL: http://arxiv.org/abs/2510.24014v2
• Date: Thu, 30 Oct 2025 05:38:02 GMT
• Title: TEXT2DB: Integration-Aware Information Extraction with Large Language Model Agents
• Authors: Yizhu Jiao, Sha Li, Sizhe Zhou, Heng Ji, Jiawei Han,
• Abstract summary: We propose a new formulation of IE TEXT2DB that emphasizes the integration of IE output and the target database.<n>We introduce a new benchmark featuring common demands such as data infilling, row population, and column addition.<n>Experiments show that OPAL can successfully adapt to diverse database schemas by generating different code plans and calling the required IE models.
• Score: 64.11547566154947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of information extraction (IE) is to extract structured knowledge from text. However, it is often not straightforward to utilize IE output due to the mismatch between the IE ontology and the downstream application needs. We propose a new formulation of IE TEXT2DB that emphasizes the integration of IE output and the target database (or knowledge base). Given a user instruction, a document set, and a database, our task requires the model to update the database with values from the document set to satisfy the user instruction. This task requires understanding user instructions for what to extract and adapting to the given DB/KB schema for how to extract on the fly. To evaluate this new task, we introduce a new benchmark featuring common demands such as data infilling, row population, and column addition. In addition, we propose an LLM agent framework OPAL (Observe-PlanAnalyze LLM) which includes an Observer component that interacts with the database, the Planner component that generates a code-based plan with calls to IE models, and the Analyzer component that provides feedback regarding code quality before execution. Experiments show that OPAL can successfully adapt to diverse database schemas by generating different code plans and calling the required IE models. We also highlight difficult cases such as dealing with large databases with complex dependencies and extraction hallucination, which we believe deserve further investigation. Source code: https://github.com/yzjiao/Text2DB

Related papers

• AskDB: An LLM Agent for Natural Language Interaction with Relational Databases [0.06524460254566904]
  We introduce AskDB, a large language model powered agent for interacting with databases through natural language.<n>AskDB supports both data analysis and administrative operations oversql databases through natural language.<n>Our results highlight the potential of AskDB as a unified and intelligent agent for relational database systems, offering an intuitive and accessible experience for end users.
  arXiv  Detail & Related papers  (2025-11-20T08:06:09Z)
• Text2Schema: Filling the Gap in Designing Database Table Structures based on Natural Language [22.15408079332362]
  People without a database background usually rely on file systems or tools such as Excel data management.<n> Database systems possess strong management capabilities, but require a high level of professional expertise from users.
  arXiv  Detail & Related papers  (2025-03-31T09:39:19Z)
• New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration [49.180693704510006]
  Referring Expression (REC) is a cross-modal task that evaluates the interplay of language understanding, image comprehension, and language-to-image grounding.<n>It serves as an essential testing ground for Multimodal Large Language Models (MLLMs)
  arXiv  Detail & Related papers  (2025-02-27T13:58:44Z)
• CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases [13.733229886643041]
  Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval and MBPP, but struggle with handling entire code repositories. Similarity-based retrieval often has low recall in complex tasks, while manual tools and APIs are typically task-specific and require expert knowledge. We introduce CodexGraph, a system that integrates LLM agents with graph database interfaces extracted from code repositories.
  arXiv  Detail & Related papers  (2024-08-07T17:13:59Z)
• TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools [51.576974932743596]
  Large Language Models (LLMs) often do not perform well on queries that require the aggregation of information across texts. TACT contains challenging instructions that demand stitching information scattered across one or more texts. We construct this dataset by leveraging an existing dataset of texts and their associated tables. We demonstrate that all contemporary LLMs perform poorly on this dataset, achieving an accuracy below 38%.
  arXiv  Detail & Related papers  (2024-06-05T20:32:56Z)
• ADELIE: Aligning Large Language Models on Information Extraction [55.60192044049083]
  Large language models (LLMs) usually fall short on information extraction tasks. In this paper, we introduce ADELIE, an aligned LLM that effectively solves various IE tasks. We show that our models achieve state-of-the-art (SoTA) performance among open-source models.
  arXiv  Detail & Related papers  (2024-05-08T12:24:52Z)
• Instruct and Extract: Instruction Tuning for On-Demand Information Extraction [86.29491354355356]
  On-Demand Information Extraction aims to fulfill the personalized demands of real-world users. We present a benchmark named InstructIE, inclusive of both automatically generated training data, as well as the human-annotated test set. Building on InstructIE, we further develop an On-Demand Information Extractor, ODIE.
  arXiv  Detail & Related papers  (2023-10-24T17:54:25Z)
• AskYourDB: An end-to-end system for querying and visualizing relational databases using natural language [0.0]
  We propose a semantic parsing approach to address the challenge of converting complex natural language into SQL. We modified state-of-the-art models, by various pre and post processing steps which make the significant part when a model is deployed in production. To make the product serviceable to businesses we added an automatic visualization framework over the queried results.
  arXiv  Detail & Related papers  (2022-10-16T13:31:32Z)
• SciREX: A Challenge Dataset for Document-Level Information Extraction [56.83748634747753]
  It is challenging to create a large-scale information extraction dataset at the document level. We introduce SciREX, a document level IE dataset that encompasses multiple IE tasks. We develop a neural model as a strong baseline that extends previous state-of-the-art IE models to document-level IE.
  arXiv  Detail & Related papers  (2020-05-01T17:30:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.