Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking

• URL: http://arxiv.org/abs/2112.01206v1
• Date: Thu, 2 Dec 2021 13:20:26 GMT
• Title: Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking
• Authors: Nianlong Gu, Yingqiang Gao, Richard H.R. Hahnloser
• Abstract summary: BM25 has been found to be a tough-to-beat approach to prefetching. In this paper, we explore prefetching with nearest neighbor search among text embeddings constructed by a hierarchical attention network. When coupled with a SciBERT reranker fine-tuned on local citation recommendation tasks, our hierarchical Attention encoder (HAtten) achieves high prefetch recall for a given number of candidates to be reranked.
• Score: 6.456347800676685
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The goal of local citation recommendation is to recommend a missing reference from the local citation context and optionally also from the global context. To balance the tradeoff between speed and accuracy of citation recommendation in the context of a large-scale paper database, a viable approach is to first prefetch a limited number of relevant documents using efficient ranking methods and then to perform a fine-grained reranking using more sophisticated models. In that vein, BM25 has been found to be a tough-to-beat approach to prefetching, which is why recent work has focused mainly on the reranking step. Even so, we explore prefetching with nearest neighbor search among text embeddings constructed by a hierarchical attention network. When coupled with a SciBERT reranker fine-tuned on local citation recommendation tasks, our hierarchical Attention encoder (HAtten) achieves high prefetch recall for a given number of candidates to be reranked. Consequently, our reranker needs to rerank fewer prefetch candidates, yet still achieves state-of-the-art performance on various local citation recommendation datasets such as ACL-200, FullTextPeerRead, RefSeer, and arXiv.

Related papers

• Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers [66.55612528039894]
  AdaQR is a framework for training query rewriting models with limited rewrite annotations from seed datasets and completely no passage label. A novel approach is proposed to assess retriever's preference for these candidates by the probability of answers conditioned on the conversational query.
  arXiv  Detail & Related papers  (2024-06-16T16:09:05Z)
• ILCiteR: Evidence-grounded Interpretable Local Citation Recommendation [31.259805200946175]
  We introduce the evidence-grounded local citation recommendation task, where the target latent space comprises evidence spans for recommending specific papers. Unlike past formulations that simply output recommendations, ILCiteR retrieves ranked lists of evidence span and recommended paper pairs. We contribute a novel dataset for the evidence-grounded local citation recommendation task and demonstrate the efficacy of our proposed conditional neural rank-ensembling approach for re-ranking evidence spans.
  arXiv  Detail & Related papers  (2024-03-13T17:38:05Z)
• Lexically-Accelerated Dense Retrieval [29.327878974130055]
  'LADR' (Lexically-Accelerated Dense Retrieval) is a simple-yet-effective approach that improves the efficiency of existing dense retrieval models. LADR consistently achieves both precision and recall that are on par with an exhaustive search on standard benchmarks.
  arXiv  Detail & Related papers  (2023-07-31T15:44:26Z)
• ReFIT: Relevance Feedback from a Reranker during Inference [109.33278799999582]
  Retrieve-and-rerank is a prevalent framework in neural information retrieval. We propose to leverage the reranker to improve recall by making it provide relevance feedback to the retriever at inference time.
  arXiv  Detail & Related papers  (2023-05-19T15:30:33Z)
• Progressive End-to-End Object Detection in Crowded Scenes [96.92416613336096]
  Previous query-based detectors suffer from two drawbacks: first, multiple predictions will be inferred for a single object, typically in crowded scenes; second, the performance saturates as the depth of the decoding stage increases. We propose a progressive predicting method to address the above issues. Specifically, we first select accepted queries to generate true positive predictions, then refine the rest noisy queries according to the previously accepted predictions. Experiments show that our method can significantly boost the performance of query-based detectors in crowded scenes.
  arXiv  Detail & Related papers  (2022-03-15T06:12:00Z)
• Recommending Multiple Positive Citations for Manuscript via Content-Dependent Modeling and Multi-Positive Triplet [6.7854900381386845]
  We propose a novel scientific paper modeling for citation recommendations, namely Multi-Positive BERT Model for Citation Recommendation (MP-BERT4CR) The proposed approach has the following advantages: First, the proposed multi-positive objectives are effective to recommend multiple positive candidates. MP-BERT4CR are also effective in retrieving the full list of co-citations, and historically low-frequent co-citation pairs compared with the prior works.
  arXiv  Detail & Related papers  (2021-11-25T04:09:31Z)
• Revisiting Deep Local Descriptor for Improved Few-Shot Classification [56.74552164206737]
  We show how one can improve the quality of embeddings by leveraging textbfDense textbfClassification and textbfAttentive textbfPooling. We suggest to pool feature maps by applying attentive pooling instead of the widely used global average pooling (GAP) to prepare embeddings for few-shot classification.
  arXiv  Detail & Related papers  (2021-03-30T00:48:28Z)
• Scene Text Detection with Selected Anchor [16.27975694546667]
  Object proposal technique with dense anchoring scheme for scene text detection was applied frequently to achieve high recall. We propose an anchor selection-based region proposal network (AS-RPN) using effective selected anchors instead of dense anchors.
  arXiv  Detail & Related papers  (2020-08-19T16:03:13Z)
• Learning Neural Textual Representations for Citation Recommendation [7.227232362460348]
  We propose a novel approach to citation recommendation using a deep sequential representation of the documents (Sentence-BERT) cascaded with Siamese and triplet networks in a submodular scoring function. To the best of our knowledge, this is the first approach to combine deep representations and submodular selection for a task of citation recommendation.
  arXiv  Detail & Related papers  (2020-07-08T12:38:50Z)
• Context-Based Quotation Recommendation [60.93257124507105]
  We propose a novel context-aware quote recommendation system. It generates a ranked list of quotable paragraphs and spans of tokens from a given source document. We conduct experiments on a collection of speech transcripts and associated news articles.
  arXiv  Detail & Related papers  (2020-05-17T17:49:53Z)
• Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning [61.32992639292889]
  Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. We introduce a new scoring method that casts a plausibility ranking task in a full-text format. We show that our method provides a much more stable training phase across random restarts.
  arXiv  Detail & Related papers  (2020-04-29T10:54:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.