SAIL: Sample-Centric In-Context Learning for Document Information Extraction

• URL: http://arxiv.org/abs/2412.17092v1
• Date: Sun, 22 Dec 2024 16:58:59 GMT
• Title: SAIL: Sample-Centric In-Context Learning for Document Information Extraction
• Authors: Jinyu Zhang, Zhiyuan You, Jize Wang, Xinyi Le,
• Abstract summary: Document Information Extraction (DIE) aims to extract structured information from Visually Rich Documents (VRDs) Previous full-training approaches have demonstrated strong performance but may struggle with generalization to unseen data. We propose Sample-centric In-context Learning (SAIL) for DIE to provide accurate guidance to pre-trained models.
• Score: 4.262937373782431
• License:
• Abstract: Document Information Extraction (DIE) aims to extract structured information from Visually Rich Documents (VRDs). Previous full-training approaches have demonstrated strong performance but may struggle with generalization to unseen data. In contrast, training-free methods leverage powerful pre-trained models like Large Language Models (LLMs) to address various downstream tasks with only a few examples. Nonetheless, training-free methods for DIE encounter two primary challenges: (1) understanding the complex relationship between layout and textual elements in VRDs, and (2) providing accurate guidance to pre-trained models. To address these challenges, we propose Sample-centric In-context Learning (SAIL) for DIE. SAIL introduces a fine-grained entity-level textual similarity to facilitate in-depth text analysis by LLMs and incorporates layout similarity to enhance the analysis of layouts in VRDs. Additionally, SAIL formulates a unified In-Context Learning (ICL) prompt template for various sample-centric examples, enabling tailored prompts that deliver precise guidance to pre-trained models for each sample. Extensive experiments on FUNSD, CORD, and SROIE benchmarks with various base models (e.g., LLMs) indicate that our method outperforms training-free baselines, even closer to the full-training methods. The results show the superiority and generalization of our method.

Related papers

• Exploring Large Language Models for Product Attribute Value Identification [25.890927969633196]
  Product attribute value identification (PAVI) involves automatically identifying attributes and their values from product information. Existing methods rely on fine-tuning pre-trained language models, such as BART and T5. This paper explores large language models (LLMs), such as LLaMA and Mistral, as data-efficient and robust alternatives for PAVI.
  arXiv  Detail & Related papers  (2024-09-19T12:09:33Z)
• RUIE: Retrieval-based Unified Information Extraction using Large Language Model [6.788855739199981]
  Unified information extraction aims to extract structured information from unstructured text. We propose RUIE (Retrieval-based Unified Information Extraction), a framework that leverages in-context learning for efficient task generalization.
  arXiv  Detail & Related papers  (2024-09-18T03:20:04Z)
• Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization [165.98557106089777]
  A key challenge is to enhance the capabilities of large language models (LLMs) amid a looming shortage of high-quality training data. Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets. We then formalize this strategy into a principled framework of Instance-Reweighted Distributionally Robust Optimization.
  arXiv  Detail & Related papers  (2024-02-22T04:10:57Z)
• One-Shot Learning as Instruction Data Prospector for Large Language Models [108.81681547472138]
  textscNuggets uses one-shot learning to select high-quality instruction data from extensive datasets. We show that instruction tuning with the top 1% of examples curated by textscNuggets substantially outperforms conventional methods employing the entire dataset.
  arXiv  Detail & Related papers  (2023-12-16T03:33:12Z)
• VILA: On Pre-training for Visual Language Models [74.08039416548209]
  We study the design options for VLM pre-training through step-by-step controllable comparisons. We build VILA, a Visual Language model family that consistently outperforms the state-of-the-art models.
  arXiv  Detail & Related papers  (2023-12-12T18:58:18Z)
• Unified Demonstration Retriever for In-Context Learning [56.06473069923567]
  Unified Demonstration Retriever (textbfUDR) is a single model to retrieve demonstrations for a wide range of tasks. We propose a multi-task list-wise ranking training framework, with an iterative mining strategy to find high-quality candidates. Experiments on 30+ tasks across 13 task families and multiple data domains show that UDR significantly outperforms baselines.
  arXiv  Detail & Related papers  (2023-05-07T16:07:11Z)
• SPEC: Summary Preference Decomposition for Low-Resource Abstractive Summarization [21.037841262371355]
  We present a framework to transfer few-shot learning processes from source corpora to the target corpus. Our methods achieve state-of-the-art performance on six diverse corpora with 30.11%/33.95%/27.51% and 26.74%/31.14%/24.48% average improvements on ROUGE-1/2/L under 10- and 100-example settings.
  arXiv  Detail & Related papers  (2023-03-24T14:07:03Z)
• ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction [56.790794611002106]
  Large language models (LLMs) have demonstrated remarkable results in various natural language processing (NLP) tasks with in-context learning. We propose a simple but effective in-context learning framework called ICL-D3IE. Specifically, we extract the most difficult and distinct segments from hard training documents as hard demonstrations.
  arXiv  Detail & Related papers  (2023-03-09T06:24:50Z)
• DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning [83.48587570246231]
  Visual Similarity plays an important role in many computer vision applications. Deep metric learning (DML) is a powerful framework for learning such similarities. We propose and study multiple complementary learning tasks, targeting conceptually different data relationships. We learn a single model to aggregate their training signals, resulting in strong generalization and state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-04-28T12:26:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.