Interpretable Coreference Resolution Evaluation Using Explicit Semantics

• URL: http://arxiv.org/abs/2605.10627v1
• Date: Mon, 11 May 2026 14:20:16 GMT
• Title: Interpretable Coreference Resolution Evaluation Using Explicit Semantics
• Abstract summary: Coreference resolution is evaluated using aggregate statistical metrics such as CoNLL-F1.<n>We introduce a semantically-enhanced evaluation framework for coreference resolution.<n>We show that our framework uncovers systematic weaknesses that remain obscured by aggregate metrics.
• Score: 35.71215348826291
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Coreference resolution is typically evaluated using aggregate statistical metrics such as CoNLL-F1, which measure structural overlap between predicted and gold clusters. While widely used, these metrics offer limited diagnostic insights, penalizing errors without revealing whether a system struggles with specific semantic categories, such as people, locations, or events, and making it difficult to interpret model capabilities or derive actionable improvements. We address this gap by introducing a semantically-enhanced evaluation framework for coreference resolution. Our approach overlays Concept and Named Entity Recognition (CNER) onto coreference outputs, assigning semantic labels to nominal mentions and propagating them to entire coreference clusters. This enables the computation of typed scores aimed at evaluating mention extraction and linking capabilities stratified by semantic class. Across our experiments on OntoNotes, LitBank, and PreCo, we show that our framework uncovers systematic weaknesses that remain obscured by aggregate metrics. Furthermore, we demonstrate that these diagnostics can be used to design targeted, low-cost data augmentation strategies, achieving measurable out-of-domain improvements.

Related papers

• External Clustering Validation by the Homogeneity-Parsimony Trade-off [0.0]
  We describe normalized scores of cluster homogeneity and parsimony that quantify this trade-off.<n>We extend the information-theoretic framework beyond Shannon entropies to derive set-matching and pair-based counterparts.<n>We demonstrate the framework's utility for feature selection and algorithm comparison.
  arXiv  Detail & Related papers  (2026-07-22T23:55:33Z)
• Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations [12.535445487099393]
  We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts.<n>We validate this matching through targeted attribute perturbations.<n>Our framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs.
  arXiv  Detail & Related papers  (2026-06-23T15:39:29Z)
• Beyond Statistical Co-occurrence: Unlocking Intrinsic Semantics for Tabular Data Clustering [52.97486694817375]
  This paper proposes Tabular-Augmented Contrastive Clustering (TagCC) to bridge the gap between dataset-specific statistics and intrinsic semantic knowledge.<n>TagCC distills underlying data semantics into textual anchors via semantic-aware transformation.<n>It is jointly optimized with a clustering objective, ensuring that the learned representations are both semantically coherent and clustering-friendly.
  arXiv  Detail & Related papers  (2026-04-13T00:25:22Z)
• Bridging the Semantic Gap for Categorical Data Clustering via Large Language Models [64.58262227709842]
  ARISE (Attention-weighted Representation with Integrated Semantic Embeddings) is presented.<n>It builds semantic-aware representations that complement the metric space of categorical data for accurate clustering.<n>Experiments on eight benchmark datasets demonstrate consistent improvements over seven representative counterparts.
  arXiv  Detail & Related papers  (2026-01-03T11:37:46Z)
• Concept Regions Matter: Benchmarking CLIP with a New Cluster-Importance Approach [20.898059440239603]
  Cluster-based Concept Importance (CCI) is a novel interpretability method.<n>CCI sets a new state of the art on faithfulness benchmarks.<n>We present a comprehensive evaluation of eighteen CLIP variants.
  arXiv  Detail & Related papers  (2025-11-17T05:01:24Z)
• SCORE: A Semantic Evaluation Framework for Generative Document Parsing [2.5101597298392098]
  Multi-modal generative document parsing systems produce semantically correct yet structurally divergent outputs.<n>Conventional metrics-CER, WER, IoU, or TEDS-misclassify such diversity as error, penalizing valid interpretations and obscuring system behavior.<n>We introduce SCORE, an interpretation-agnostic framework that integrates (i) adjusted edit distance for robust content fidelity, (ii) token-level diagnostics to distinguish hallucinations from omissions, (iii) table evaluation with spatial tolerance and semantic alignment, and (iv) hierarchy-aware consistency checks.
  arXiv  Detail & Related papers  (2025-09-16T16:06:19Z)
• SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection [70.23196257213829]
  We propose a scalable and reliable Semantic-level Evaluation framework for Open domain Event detection.<n>Our proposed framework first constructs a scalable evaluation benchmark that currently includes 564 event types covering 7 major domains.<n>We then leverage large language models (LLMs) as automatic evaluation agents to compute a semantic F1-score, incorporating fine-grained definitions of semantically similar labels.
  arXiv  Detail & Related papers  (2025-03-05T09:37:05Z)
• Predict, Cluster, Refine: A Joint Embedding Predictive Self-Supervised Framework for Graph Representation Learning [0.0]
  Graph representation learning has emerged as a cornerstone for tasks like node classification and link prediction.<n>Current self-supervised learning (SSL) methods face challenges such as computational inefficiency, reliance on contrastive objectives, and representation collapse.<n>We propose a novel joint embedding predictive framework for graph SSL that eliminates contrastive objectives and negative sampling while preserving semantic and structural information.
  arXiv  Detail & Related papers  (2025-02-02T07:42:45Z)
• Open-Vocabulary Segmentation with Semantic-Assisted Calibration [68.41025728960176]
  We study open-vocabulary segmentation (OVS) through calibrating in-vocabulary and domain-biased embedding space with contextual prior of CLIP.<n>We present a Semantic-assisted CAlibration Network (SCAN) to achieve state-of-the-art performance on open-vocabulary segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-12-07T07:00:09Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)
• Towards Uncovering the Intrinsic Data Structures for Unsupervised Domain Adaptation using Structurally Regularized Deep Clustering [119.88565565454378]
  Unsupervised domain adaptation (UDA) is to learn classification models that make predictions for unlabeled data on a target domain. We propose a hybrid model of Structurally Regularized Deep Clustering, which integrates the regularized discriminative clustering of target data with a generative one. Our proposed H-SRDC outperforms all the existing methods under both the inductive and transductive settings.
  arXiv  Detail & Related papers  (2020-12-08T08:52:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.