Adapting PromptORE for Modern History: Information Extraction from Hispanic Monarchy Documents of the XVIth Century

• URL: http://arxiv.org/abs/2406.00027v1
• Date: Fri, 24 May 2024 13:39:47 GMT
• Title: Adapting PromptORE for Modern History: Information Extraction from Hispanic Monarchy Documents of the XVIth Century
• Authors: Hèctor Loopez Hidalgo, Michel Boeglin, David Kahn, Josiane Mothe, Diego Ortiz, David Panzoli,
• Abstract summary: We introduce an adaptation of PromptORE to extract relations from specialized documents, namely digital transcripts of trials from the Spanish Inquisition. Our approach involves fine-tuning transformer models with their pretraining objective on the data they will perform inference. Our results show a substantial improvement in accuracy -up to a 50% improvement with our Biased PromptORE models.
• Score: 2.490441444378203
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Semantic relations among entities are a widely accepted method for relation extraction. PromptORE (Prompt-based Open Relation Extraction) was designed to improve relation extraction with Large Language Models on generalistic documents. However, it is less effective when applied to historical documents, in languages other than English. In this study, we introduce an adaptation of PromptORE to extract relations from specialized documents, namely digital transcripts of trials from the Spanish Inquisition. Our approach involves fine-tuning transformer models with their pretraining objective on the data they will perform inference. We refer to this process as "biasing". Our Biased PromptORE addresses complex entity placements and genderism that occur in Spanish texts. We solve these issues by prompt engineering. We evaluate our method using Encoder-like models, corroborating our findings with experts' assessments. Additionally, we evaluate the performance using a binomial classification benchmark. Our results show a substantial improvement in accuracy -up to a 50% improvement with our Biased PromptORE models in comparison to the baseline models using standard PromptORE.

Related papers

• Likelihood as a Performance Gauge for Retrieval-Augmented Generation [78.28197013467157]
  We show that likelihoods serve as an effective gauge for language model performance. We propose two methods that use question likelihood as a gauge for selecting and constructing prompts that lead to better performance.
  arXiv  Detail & Related papers  (2024-11-12T13:14:09Z)
• UniArk: Improving Generalisation and Consistency for Factual Knowledge Extraction through Debiasing [19.2764682793582]
  We show the inherent misalignment between pre-training and downstream tuning objectives in language models for probing knowledge. We propose an adapter-based framework, UniArk, for generalised and consistent factual knowledge extraction.
  arXiv  Detail & Related papers  (2024-04-01T17:22:07Z)
• Enhancing Retrieval Processes for Language Generation with Augmented Queries [0.0]
  This research focuses on addressing this issue through Retrieval-Augmented Generation (RAG), a technique that guides models to give accurate responses based on real facts. To overcome scalability issues, the study explores connecting user queries with sophisticated language models such as BERT and Orca2. The empirical results indicate a significant improvement in the initial language model's performance under RAG.
  arXiv  Detail & Related papers  (2024-02-06T13:19:53Z)
• Fairness-guided Few-shot Prompting for Large Language Models [93.05624064699965]
  In-context learning can suffer from high instability due to variations in training examples, example order, and prompt formats. We introduce a metric to evaluate the predictive bias of a fixed prompt against labels or a given attributes. We propose a novel search strategy based on the greedy search to identify the near-optimal prompt for improving the performance of in-context learning.
  arXiv  Detail & Related papers  (2023-03-23T12:28:25Z)
• Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval [109.62363167257664]
  We propose a generative model for learning multilingual text embeddings. Our model operates on parallel data in $N$ languages. We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
  arXiv  Detail & Related papers  (2022-12-21T02:41:40Z)
• Enriching Relation Extraction with OpenIE [70.52564277675056]
  Relation extraction (RE) is a sub-discipline of information extraction (IE) In this work, we explore how recent approaches for open information extraction (OpenIE) may help to improve the task of RE. Our experiments over two annotated corpora, KnowledgeNet and FewRel, demonstrate the improved accuracy of our enriched models.
  arXiv  Detail & Related papers  (2022-12-19T11:26:23Z)
• Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking [56.80065604034095]
  We introduce a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. To evaluate our different integration strategies, we transform four existing information retrieval datasets into the relevance feedback scenario.
  arXiv  Detail & Related papers  (2022-10-19T16:19:37Z)
• Query Expansion Using Contextual Clue Sampling with Language Models [69.51976926838232]
  We propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context. Our lexical matching based approach achieves a similar top-5/top-20 retrieval accuracy and higher top-100 accuracy compared with the well-established dense retrieval model DPR. For end-to-end QA, the reader model also benefits from our method and achieves the highest Exact-Match score against several competitive baselines.
  arXiv  Detail & Related papers  (2022-10-13T15:18:04Z)
• Entity and Evidence Guided Relation Extraction for DocRED [33.69481141963074]
  We pro-pose a joint training frameworkE2GRE(Entity and Evidence Guided Relation Extraction)for this task. We introduce entity-guided sequences as inputs to a pre-trained language model (e.g. BERT, RoBERTa) These entity-guided sequences help a pre-trained language model (LM) to focus on areas of the document related to the entity. We evaluate our E2GRE approach on DocRED, a recently released large-scale dataset for relation extraction.
  arXiv  Detail & Related papers  (2020-08-27T17:41:23Z)
• Cross-lingual Information Retrieval with BERT [8.052497255948046]
  We explore the use of the popular bidirectional language model, BERT, to model and learn the relevance between English queries and foreign-language documents. A deep relevance matching model based on BERT is introduced and trained by finetuning a pretrained multilingual BERT model with weak supervision. Experimental results of the retrieval of Lithuanian documents against short English queries show that our model is effective and outperforms the competitive baseline approaches.
  arXiv  Detail & Related papers  (2020-04-24T23:32:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.