CiteBART: Learning to Generate Citations for Local Citation Recommendation

• URL: http://arxiv.org/abs/2412.17534v2
• Date: Wed, 09 Apr 2025 20:23:16 GMT
• Title: CiteBART: Learning to Generate Citations for Local Citation Recommendation
• Authors: Ege Yiğit Çelik, Selma Tekir,
• Abstract summary: This paper introduces citation-specific pre-training within an encoder-decoder architecture.<n>Author-date citation tokens are masked to learn to reconstruct them to fulfill local citation recommendation (LCR)<n>CiteBART-Global achieves state-of-the-art performance on LCR benchmarks except for the FullTextPeerRead dataset.
• Score: 0.138120109831448
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Local citation recommendation (LCR) suggests a set of papers for a citation placeholder within a given context. The task has evolved as generative approaches have become more promising than the traditional pre-fetch and re-rank-based state-of-the-art approaches. This paper introduces citation-specific pre-training within an encoder-decoder architecture, where author-date citation tokens are masked to learn to reconstruct them to fulfill LCR. There are two variants for this pre-training. In the local context-only base scheme (CiteBART-Base), the citation token in a local context is masked to learn to predict the citation. The global version (CiteBART-Global) extends the local context with the citing paper's title and abstract to enrich the learning signal. CiteBART-Global achieves state-of-the-art performance on LCR benchmarks except for the FullTextPeerRead dataset, which is quite small to see the advantage of generative pre-training. The effect is significant in the larger benchmarks, e.g., Refseer and ArXiv., with the Refseer benchmark-trained model emerging as the best-performing model. We perform comprehensive experiments, including an ablation study, a qualitative analysis, and a taxonomy of hallucinations with detailed statistics. Our analyses confirm that CiteBART-Global has a cross-dataset generalization capability; the macro hallucination rate (MaHR) at the top-3 predictions is 4\%, and when the ground-truth is in the top-k prediction list, the hallucination tendency in the other predictions drops significantly.

Related papers

• Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models [53.17363502535395]
  Trustworthy language models should provide both correct and verifiable answers.<n>Current systems insert citations by querying an external retriever at inference time.<n>We propose Active Indexing, which continually pretrains on synthetic QA pairs.
  arXiv  Detail & Related papers  (2025-06-21T04:48:05Z)
• CiteEval: Principle-Driven Citation Evaluation for Source Attribution [38.24323805177938]
  CiteEval is a citation evaluation framework driven by principles focusing on fine-grained citation assessment.<n>CiteBench is a benchmark with high-quality human annotations on citation quality.<n>CiteEval-Auto is a suite of model-based metrics that exhibit strong correlation with human judgments.
  arXiv  Detail & Related papers  (2025-06-02T16:15:34Z)
• HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction [14.731720495144112]
  We introduce the novel concept of core citation, which identifies the critical references that go beyond superficial mentions. We propose $textbfHLM-Cite, a $textbfH$ybrid $textbfL$anguage $textbfM$odel workflow for citation prediction. We evaluate HLM-Cite across 19 scientific fields, demonstrating a 17.6% performance improvement comparing SOTA methods.
  arXiv  Detail & Related papers  (2024-10-10T10:46:06Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• ALiiCE: Evaluating Positional Fine-grained Citation Generation [54.19617927314975]
  We propose ALiiCE, the first automatic evaluation framework for fine-grained citation generation. Our framework first parses the sentence claim into atomic claims via dependency analysis and then calculates citation quality at the atomic claim level. We evaluate the positional fine-grained citation generation performance of several Large Language Models on two long-form QA datasets.
  arXiv  Detail & Related papers  (2024-06-19T09:16:14Z)
• Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias [1.7812428873698407]
  Citation practices are crucial in shaping the structure of scientific knowledge, yet they are often influenced by contemporary norms and biases. The emergence of Large Language Models (LLMs) introduces a new dynamic to these practices. Here, we analyze these characteristics in an experiment using a dataset from AAAI, NeurIPS, ICML, and ICLR.
  arXiv  Detail & Related papers  (2024-05-24T17:34:32Z)
• ILCiteR: Evidence-grounded Interpretable Local Citation Recommendation [31.259805200946175]
  We introduce the evidence-grounded local citation recommendation task, where the target latent space comprises evidence spans for recommending specific papers. Unlike past formulations that simply output recommendations, ILCiteR retrieves ranked lists of evidence span and recommended paper pairs. We contribute a novel dataset for the evidence-grounded local citation recommendation task and demonstrate the efficacy of our proposed conditional neural rank-ensembling approach for re-ranking evidence spans.
  arXiv  Detail & Related papers  (2024-03-13T17:38:05Z)
• Contextualizing Generated Citation Texts [11.531517736126657]
  We propose a simple modification to the citation text generation task. The generation target is not only the citation itself, but the entire context window, including the target citation.
  arXiv  Detail & Related papers  (2024-02-28T05:24:21Z)
• CiteBench: A benchmark for Scientific Citation Text Generation [69.37571393032026]
  CiteBench is a benchmark for citation text generation. We make the code for CiteBench publicly available at https://github.com/UKPLab/citebench.
  arXiv  Detail & Related papers  (2022-12-19T16:10:56Z)
• Controllable Citation Sentence Generation with Language Models [11.186252009101077]
  We propose to integrate the manuscript context, the context of the referenced paper, and the desired control attributes into a structured template and use it to fine-tune a language model (LM) via next-token prediction. The proposed workflow harmoniously combines citation attribute suggestion and conditional citation generation into one LM, allowing for better user control.
  arXiv  Detail & Related papers  (2022-11-14T01:54:08Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)
• QuoteR: A Benchmark of Quote Recommendation for Writing [80.83859760380616]
  We build a large and fully open quote recommendation dataset called QuoteR. We conduct an extensive evaluation of existing quote recommendation methods on QuoteR. We propose a new quote recommendation model that significantly outperforms previous methods on all three parts of QuoteR.
  arXiv  Detail & Related papers  (2022-02-26T14:01:44Z)
• Towards generating citation sentences for multiple references with intent control [86.53829532976303]
  We build a novel generation model with the Fusion-in-Decoder approach to cope with multiple long inputs. Experiments demonstrate that the proposed approaches provide much more comprehensive features for generating citation sentences.
  arXiv  Detail & Related papers  (2021-12-02T15:32:24Z)
• Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-based Reranking [6.456347800676685]
  BM25 has been found to be a tough-to-beat approach to prefetching. In this paper, we explore prefetching with nearest neighbor search among text embeddings constructed by a hierarchical attention network. When coupled with a SciBERT reranker fine-tuned on local citation recommendation tasks, our hierarchical Attention encoder (HAtten) achieves high prefetch recall for a given number of candidates to be reranked.
  arXiv  Detail & Related papers  (2021-12-02T13:20:26Z)
• Context-Aware Legal Citation Recommendation using Deep Learning [4.157772749568094]
  Lawyers and judges spend a large amount of time researching the proper legal authority to cite while drafting decisions. We develop a citation recommendation tool that can help improve efficiency in the process of opinion drafting.
  arXiv  Detail & Related papers  (2021-06-20T23:23:11Z)
• Citations are not opinions: a corpus linguistics approach to understanding how citations are made [0.0]
  Key issue in citation content analysis is looking for linguistic structures that characterize distinct classes of citations. In this study, we start with a large sample of a pre-classified citation corpus, 2 million citations from each class of the scite Smart Citation dataset. By generating comparison tables for each citation type, we present a number of interesting linguistic features that uniquely characterize citation type.
  arXiv  Detail & Related papers  (2021-04-16T12:52:27Z)
• Enhancing Scientific Papers Summarization with Citation Graph [78.65955304229863]
  We redefine the task of scientific papers summarization by utilizing their citation graph. We construct a novel scientific papers summarization dataset Semantic Scholar Network (SSN) which contains 141K research papers in different domains. Our model can achieve competitive performance when compared with the pretrained models.
  arXiv  Detail & Related papers  (2021-04-07T11:13:35Z)
• Towards Making the Most of Context in Neural Machine Translation [112.9845226123306]
  We argue that previous research did not make a clear use of the global context. We propose a new document-level NMT framework that deliberately models the local context of each sentence.
  arXiv  Detail & Related papers  (2020-02-19T03:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.