Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem

• URL: http://arxiv.org/abs/2204.05990v1
• Date: Tue, 12 Apr 2022 17:55:22 GMT
• Title: Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem
• Authors: Khalil Mrini, Shaoliang Nie, Jiatao Gu, Sinong Wang, Maziar Sanjabi, Hamed Firooz
• Abstract summary: We propose an autoregressive entity linking model, that is trained with two auxiliary tasks, and learns to re-rank generated samples at inference time. We show through ablation studies that each of the two auxiliary tasks increases performance, and that re-ranking is an important factor to the increase.
• Score: 46.028180604304985
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose an autoregressive entity linking model, that is trained with two auxiliary tasks, and learns to re-rank generated samples at inference time. Our proposed novelties address two weaknesses in the literature. First, a recent method proposes to learn mention detection and then entity candidate selection, but relies on predefined sets of candidates. We use encoder-decoder autoregressive entity linking in order to bypass this need, and propose to train mention detection as an auxiliary task instead. Second, previous work suggests that re-ranking could help correct prediction errors. We add a new, auxiliary task, match prediction, to learn re-ranking. Without the use of a knowledge base or candidate sets, our model sets a new state of the art in two benchmark datasets of entity linking: COMETA in the biomedical domain, and AIDA-CoNLL in the news domain. We show through ablation studies that each of the two auxiliary tasks increases performance, and that re-ranking is an important factor to the increase. Finally, our low-resource experimental results suggest that performance on the main task benefits from the knowledge learned by the auxiliary tasks, and not just from the additional training data.

Related papers

• Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection [7.864892339833315]
  We propose a novel task-driven top-down framework for joint moment retrieval and highlight detection. The framework introduces a task-decoupled unit to capture task-specific and common representations. Comprehensive experiments and in-depth ablation studies on QVHighlights, TVSum, and Charades-STA datasets corroborate the effectiveness and flexibility of the proposed framework.
  arXiv  Detail & Related papers  (2024-04-14T14:06:42Z)
• List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation [80.12531449946655]
  We propose a Reranking-Truncation joint model (GenRT) that can perform the two tasks concurrently. GenRT integrates reranking and truncation via generative paradigm based on encoder-decoder architecture. Our method achieves SOTA performance on both reranking and truncation tasks for web search and retrieval-augmented LLMs.
  arXiv  Detail & Related papers  (2024-02-05T06:52:53Z)
• Divide and Conquer: Hybrid Pre-training for Person Search [40.13016375392472]
  We propose a hybrid pre-training framework specifically designed for person search using sub-task data only. Our model can achieve significant improvements across diverse protocols, such as person search method, fine-tuning data, pre-training data and model backbone. Our code and pre-trained models are released for plug-and-play usage to the person search community.
  arXiv  Detail & Related papers  (2023-12-13T08:33:50Z)
• Neural Coreference Resolution based on Reinforcement Learning [53.73316523766183]
  Coreference resolution systems need to solve two subtasks. One task is to detect all of the potential mentions, the other is to learn the linking of an antecedent for each possible mention. We propose a reinforcement learning actor-critic-based neural coreference resolution system.
  arXiv  Detail & Related papers  (2022-12-18T07:36:35Z)
• Composite Learning for Robust and Effective Dense Predictions [81.2055761433725]
  Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. We find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
  arXiv  Detail & Related papers  (2022-10-13T17:59:16Z)
• Task Compass: Scaling Multi-task Pre-training with Task Prefix [122.49242976184617]
  Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. We propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. Our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.
  arXiv  Detail & Related papers  (2022-10-12T15:02:04Z)
• Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks [0.0]
  Recent work on few-shot learning shows that quality of learned representations plays an important role in few-shot classification performance. On the other hand, the goal of self-supervised learning is to recover useful semantic information of the data without the use of class labels. We exploit the complementarity of both paradigms via a multi-task framework where we leverage recent self-supervised methods as auxiliary tasks.
  arXiv  Detail & Related papers  (2021-01-24T23:21:43Z)
• Curriculum-Meta Learning for Order-Robust Continual Relation Extraction [12.494209368988253]
  We propose a novel curriculum-meta learning method to tackle the challenges of continual relation extraction. We combine meta learning and curriculum learning to quickly adapt model parameters to a new task. We present novel difficulty-based metrics to quantitatively measure the extent of order-sensitivity of a given model.
  arXiv  Detail & Related papers  (2021-01-06T08:52:34Z)
• Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing [85.35582118010608]
  Task-oriented semantic parsing is a critical component of virtual assistants. Recent advances in deep learning have enabled several approaches to successfully parse more complex queries. We propose a novel method that outperforms a supervised neural model at a 10-fold data reduction.
  arXiv  Detail & Related papers  (2020-10-07T17:47:53Z)
• Sequential Transfer in Reinforcement Learning with a Generative Model [48.40219742217783]
  We show how to reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones. We derive PAC bounds on its sample complexity which clearly demonstrate the benefits of using this kind of prior knowledge. We empirically verify our theoretical findings in simple simulated domains.
  arXiv  Detail & Related papers  (2020-07-01T19:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.