Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities

• URL: http://arxiv.org/abs/2205.11438v1
• Date: Mon, 23 May 2022 16:30:20 GMT
• Title: Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities
• Abstract summary: We present an approach to entity and event coreference resolution utilizing contrastive representation learning. Our method achieves state-of-the-art results on a number of key metrics on the ECB+ corpus.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identifying related entities and events within and across documents is fundamental to natural language understanding. We present an approach to entity and event coreference resolution utilizing contrastive representation learning. Earlier state-of-the-art methods have formulated this problem as a binary classification problem and leveraged large transformers in a cross-encoder architecture to achieve their results. For large collections of documents and corresponding set of $n$ mentions, the necessity of performing $n^{2}$ transformer computations in these earlier approaches can be computationally intensive. We show that it is possible to reduce this burden by applying contrastive learning techniques that only require $n$ transformer computations at inference time. Our method achieves state-of-the-art results on a number of key metrics on the ECB+ corpus and is competitive on others.

Related papers

• URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding [55.45331924836242]
  We present URaG, a framework that Unifies Retrieval and Generation within a single MLLM.<n>We show that URaG achieves state-of-the-art performance while reducing computational overhead by 44-56%.
  arXiv  Detail & Related papers  (2025-11-13T17:54:09Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive. LCD can distort the global distribution over strings, sampling tokens based only on local information. We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• GeAR: Generation Augmented Retrieval [82.20696567697016]
  Document retrieval techniques form the foundation for the development of large-scale information systems. The prevailing methodology is to construct a bi-encoder and compute the semantic similarity. We propose a new method called $textbfGe$neration that incorporates well-designed fusion and decoding modules.
  arXiv  Detail & Related papers  (2025-01-06T05:29:00Z)
• Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
  Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification. In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction. Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
  arXiv  Detail & Related papers  (2024-10-24T09:09:20Z)
• Contextual Document Embeddings [77.22328616983417]
  We propose two complementary methods for contextualized document embeddings. First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss. Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
  arXiv  Detail & Related papers  (2024-10-03T14:33:34Z)
• Generative Retrieval Meets Multi-Graded Relevance [104.75244721442756]
  We introduce a framework called GRaded Generative Retrieval (GR$2$) GR$2$ focuses on two key components: ensuring relevant and distinct identifiers, and implementing multi-graded constrained contrastive training. Experiments on datasets with both multi-graded and binary relevance demonstrate the effectiveness of GR$2$.
  arXiv  Detail & Related papers  (2024-09-27T02:55:53Z)
• Equipping Transformer with Random-Access Reading for Long-Context Understanding [9.433800833564279]
  Long-context modeling presents a significant challenge for transformer-based large language models. We propose a novel reading strategy that enables transformers to efficiently process long documents without examining every token.
  arXiv  Detail & Related papers  (2024-05-21T21:41:07Z)
• Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs [75.40636935415601]
  Deep learning often faces the challenge of efficiently processing dynamic inputs, such as sensor data or user inputs. We take an incremental computing approach, looking to reuse calculations as the inputs change. We apply this approach to the transformers architecture, creating an efficient incremental inference algorithm with complexity proportional to the fraction of modified inputs.
  arXiv  Detail & Related papers  (2023-07-27T16:30:27Z)
• $2 * n$ is better than $n^2$: Decomposing Event Coreference Resolution into Two Tractable Problems [3.777476692151661]
  Event Coreference Resolution (ECR) is the task of linking mentions of the same event either within or across documents. We show that we get comparable results to the state of the art on two popular ECR datasets. We also analyze the mention pairs that are "hard" to accurately classify as coreferent or non-coreferent.
  arXiv  Detail & Related papers  (2023-05-09T05:33:32Z)
• Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference [22.497877069528087]
  Event and entity coreference resolution across documents vastly increases the number of candidate mentions, making it intractable to do the full $n2$ pairwise comparisons. Existing approaches simplify by considering coreference only within document clusters, but this fails to handle inter-cluster coreference. We draw on an insight from discourse coherence theory: potential coreferences are constrained by the reader's discourse focus. Our approach achieves state-of-the-art results for both events and entities on the ECB+, Gun Violence, Football Coreference, and Cross-Domain Cross-Document Coreference corpora.
  arXiv  Detail & Related papers  (2021-10-11T15:41:47Z)
• FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning [64.32306537419498]
  We propose a novel learned feature-based refinement and augmentation method that produces a varied set of complex transformations. These transformations also use information from both within-class and across-class representations that we extract through clustering. We demonstrate that our method is comparable to current state of art for smaller datasets while being able to scale up to larger datasets.
  arXiv  Detail & Related papers  (2020-07-16T17:55:31Z)
• Propositionalization and Embeddings: Two Sides of the Same Coin [0.0]
  This paper outlines some of the modern data processing techniques used in relational learning. It focuses on the propositionalization and embedding data transformation approaches. We present two efficient implementations of the unifying methodology.
  arXiv  Detail & Related papers  (2020-06-08T08:33:21Z)
• General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference [34.47592026375839]
  We show that some of the computational cost during inference can be amortized over the different tasks using a shared text encoder. We also compare approaches for training such an encoder and show that encoders pre-trained over multiple tasks generalize well to unseen tasks.
  arXiv  Detail & Related papers  (2020-04-29T16:11:26Z)
• Pre-training Tasks for Embedding-based Large-scale Retrieval [68.01167604281578]
  We consider the large-scale query-document retrieval problem. Given a query (e.g., a question), return the set of relevant documents from a large document corpus. We show that the key ingredient of learning a strong embedding-based Transformer model is the set of pre-training tasks.
  arXiv  Detail & Related papers  (2020-02-10T16:44:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.