Training LayoutLM from Scratch for Efficient Named-Entity Recognition in the Insurance Domain

• URL: http://arxiv.org/abs/2412.09341v1
• Date: Thu, 12 Dec 2024 15:09:44 GMT
• Title: Training LayoutLM from Scratch for Efficient Named-Entity Recognition in the Insurance Domain
• Authors: Benno Uthayasooriyar, Antoine Ly, Franck Vermet, Caio Corro,
• Abstract summary: Generic pre-trained neural networks may struggle to produce good results in specialized domains like finance and insurance. This is due to a domain mismatch between training data and downstream tasks, as in-domain data are often scarce due to privacy constraints. We show that using domain-relevant documents improves results on a named-entity recognition problem using an anonymized dataset of insurance-related financial documents.
• Score: 6.599755599064449
• License:
• Abstract: Generic pre-trained neural networks may struggle to produce good results in specialized domains like finance and insurance. This is due to a domain mismatch between training data and downstream tasks, as in-domain data are often scarce due to privacy constraints. In this work, we compare different pre-training strategies for LayoutLM. We show that using domain-relevant documents improves results on a named-entity recognition (NER) problem using a novel dataset of anonymized insurance-related financial documents called Payslips. Moreover, we show that we can achieve competitive results using a smaller and faster model.

Related papers

• Federated Unsupervised Domain Adaptation for Face Recognition [26.336693850812118]
  We propose federated unsupervised domain adaptation for face recognition, FedFR. For unlabeled data in the target domain, we enhance a clustering algorithm with distance constrain to improve the quality of predicted pseudo labels. We also propose a new domain constraint loss to regularize source domain training in federated learning.
  arXiv  Detail & Related papers  (2022-04-09T04:02:03Z)
• Domain Adaptive Semantic Segmentation without Source Data [50.18389578589789]
  We investigate domain adaptive semantic segmentation without source data, which assumes that the model is pre-trained on the source domain. We propose an effective framework for this challenging problem with two components: positive learning and negative learning. Our framework can be easily implemented and incorporated with other methods to further enhance the performance.
  arXiv  Detail & Related papers  (2021-10-13T04:12:27Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Towards Unsupervised Domain Adaptation for Deep Face Recognition under Privacy Constraints via Federated Learning [33.33475702665153]
  We propose a novel unsupervised federated face recognition approach (FedFR) FedFR improves the performance in the target domain by iteratively aggregating knowledge from the source domain through federated learning. It protects data privacy by transferring models instead of raw data between domains.
  arXiv  Detail & Related papers  (2021-05-17T04:24:25Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training [67.71228426496013]
  We show that using target domain data during pre-training leads to large performance improvements across a variety of setups. We find that pre-training on multiple domains improves performance generalization on domains not seen during training.
  arXiv  Detail & Related papers  (2021-04-02T12:53:15Z)
• CrossNER: Evaluating Cross-Domain Named Entity Recognition [47.9831214875796]
  Cross-domain named entity recognition models are able to cope with the scarcity issue of NER samples in target domains. Most of the existing NER benchmarks lack domain-specialized entity types or do not focus on a certain domain, leading to a less effective cross-domain evaluation. We introduce a cross-domain NER dataset (CrossNER), a fully-labeled collection of NER data spanning over five diverse domains with specialized entity categories for different domains.
  arXiv  Detail & Related papers  (2020-12-08T11:31:55Z)
• CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search [89.48123965553098]
  This paper presents a search system to alleviate the special domain adaption problem. The system utilizes the domain-adaptive pretraining and few-shot learning technologies to help neural rankers mitigate the domain discrepancy. Our system performs the best among the non-manual runs in Round 2 of the TREC-COVID task.
  arXiv  Detail & Related papers  (2020-11-03T09:10:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.