Recall, Expand and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing

• URL: http://arxiv.org/abs/2212.09125v1
• Date: Sun, 18 Dec 2022 16:42:52 GMT
• Title: Recall, Expand and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing
• Authors: Chengyue Jiang, Wenyang Hui, Yong Jiang, Xiaobin Wang, Pengjun Xie, Kewei Tu
• Abstract summary: State-of-the-art (SOTA) methods use the cross-encoder (CE) based architecture. We use a novel model called MCCE to concurrently encode and score these K candidates. We also found MCCE is very effective in fine-grained (130 types) and coarse-grained (9 types) entity typing.
• Score: 46.85183839946139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ultra-fine entity typing (UFET) predicts extremely free-formed types (e.g., president, politician) of a given entity mention (e.g., Joe Biden) in context. State-of-the-art (SOTA) methods use the cross-encoder (CE) based architecture. CE concatenates the mention (and its context) with each type and feeds the pairs into a pretrained language model (PLM) to score their relevance. It brings deeper interaction between mention and types to reach better performance but has to perform N (type set size) forward passes to infer types of a single mention. CE is therefore very slow in inference when the type set is large (e.g., N = 10k for UFET). To this end, we propose to perform entity typing in a recall-expand-filter manner. The recall and expand stages prune the large type set and generate K (K is typically less than 256) most relevant type candidates for each mention. At the filter stage, we use a novel model called MCCE to concurrently encode and score these K candidates in only one forward pass to obtain the final type prediction. We investigate different variants of MCCE and extensive experiments show that MCCE under our paradigm reaches SOTA performance on ultra-fine entity typing and is thousands of times faster than the cross-encoder. We also found MCCE is very effective in fine-grained (130 types) and coarse-grained (9 types) entity typing. Our code is available at \url{https://github.com/modelscope/AdaSeq/tree/master/examples/MCCE}.

Related papers

• Prototypical Hash Encoding for On-the-Fly Fine-Grained Category Discovery [65.16724941038052]
  Category-aware Prototype Generation (CPG) and Discrimi Category 5.3% (DCE) are proposed. CPG enables the model to fully capture the intra-category diversity by representing each category with multiple prototypes. DCE boosts the discrimination ability of hash code with the guidance of the generated category prototypes.
  arXiv  Detail & Related papers  (2024-10-24T23:51:40Z)
• Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing [10.08153231108538]
  We present CASENT, a seq2seq model designed for ultra-fine entity typing. Our model takes an entity mention as input and employs constrained beam search to generate multiple types autoregressively. Our method outperforms the previous state-of-the-art in terms of F1 score and calibration error, while achieving an inference speedup of over 50 times.
  arXiv  Detail & Related papers  (2023-11-01T20:39:12Z)
• ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models [24.867534196627222]
  We introduce ArcheType, a simple, practical method for context sampling, prompt serialization, model querying, and label remapping. We establish a new state-of-the-art performance on zero-shot CTA benchmarks.
  arXiv  Detail & Related papers  (2023-10-27T15:31:22Z)
• EnCore: Fine-Grained Entity Typing by Pre-Training Entity Encoders on Coreference Chains [22.469469997734965]
  We propose to pre-training an entity encoder such that embeddings of coreferring entities are more similar to each other than to the embeddings of other entities. We show that this problem can be addressed by using a simple trick: we only consider coreference links that are predicted by two different off-the-shelf systems.
  arXiv  Detail & Related papers  (2023-05-22T11:11:59Z)
• TypeT5: Seq2seq Type Inference using Static Analysis [51.153089609654174]
  We present a new type inference method that treats type prediction as a code infilling task. Our method uses static analysis to construct dynamic contexts for each code element whose type signature is to be predicted by the model. We also propose an iterative decoding scheme that incorporates previous type predictions in the model's input context.
  arXiv  Detail & Related papers  (2023-03-16T23:48:00Z)
• Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field [47.22366788848256]
  We use an undirected graphical model called pairwise conditional random field (PCRF) to formulate the UFET problem. We use various modern backbones for entity typing to compute unary potentials and derive pairwise potentials from type phrase representations. We use mean-field variational inference for efficient type inference on very large type sets and unfold it as a neural network module to enable end-to-end training.
  arXiv  Detail & Related papers  (2022-12-03T09:49:15Z)
• Multilingual Autoregressive Entity Linking [49.35994386221958]
  mGENRE is a sequence-to-sequence system for the Multilingual Entity Linking problem. For a mention in a given language, mGENRE predicts the name of the target entity left-to-right, token-by-token. We show the efficacy of our approach through extensive evaluation including experiments on three popular MEL benchmarks.
  arXiv  Detail & Related papers  (2021-03-23T13:25:55Z)
• Autoregressive Entity Retrieval [55.38027440347138]
  Entities are at the center of how we represent and aggregate knowledge. The ability to retrieve such entities given a query is fundamental for knowledge-intensive tasks such as entity linking and open-domain question answering. We propose GENRE, the first system that retrieves entities by generating their unique names, left to right, token-by-token in an autoregressive fashion.
  arXiv  Detail & Related papers  (2020-10-02T10:13:31Z)
• A Chinese Corpus for Fine-grained Entity Typing [34.93317177668996]
  We introduce a corpus for Chinese fine-grained entity typing that contains 4,800 mentions manually labeled through crowdsourcing. To make our dataset useful in more possible scenarios, we also categorize all the fine-grained types into 10 general types. We also show the possibility of improving Chinese fine-grained entity typing through cross-lingual transfer learning.
  arXiv  Detail & Related papers  (2020-04-19T11:53:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.