Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationales

• URL: http://arxiv.org/abs/2604.21667v1
• Date: Thu, 23 Apr 2026 13:30:12 GMT
• Title: Fine-Grained Perspectives: Modeling Explanations with Annotator-Specific Rationales
• Abstract summary: We propose a framework for jointly modeling annotator-specific label prediction and corresponding explanations.<n>Using a dataset with disaggregated natural language inference (NLI) annotations and annotator-provided explanations, we condition predictions on both annotator identity and demographic metadata.<n>We introduce two explainer architectures: a post-hoc prompt-based explainer and a prefixed bridge explainer that transfers annotator-conditioned representations directly into a generative model.
• Score: 5.100843711447644
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Beyond exploring disaggregated labels for modeling perspectives, annotator rationales provide fine-grained signals of individual perspectives. In this work, we propose a framework for jointly modeling annotator-specific label prediction and corresponding explanations, fine-tuned on the annotators' provided rationales. Using a dataset with disaggregated natural language inference (NLI) annotations and annotator-provided explanations, we condition predictions on both annotator identity and demographic metadata through a representation-level User Passport mechanism. We further introduce two explainer architectures: a post-hoc prompt-based explainer and a prefixed bridge explainer that transfers annotator-conditioned classifier representations directly into a generative model. This design enables explanation generation aligned with individual annotator perspectives. Our results show that incorporating explanation modeling substantially improves predictive performance over a baseline annotator-aware classifier, with the prefixed bridge approach achieving more stable label alignment and higher semantic consistency, while the post-hoc approach yields stronger lexical similarity. These findings indicate that modeling explanations as expressions of fine-grained perspective provides a richer and more faithful representation of disagreement. The proposed approaches advance perspectivist modeling by integrating annotator-specific rationales into both predictive and generative components.

Related papers

• Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning [4.570003973862485]
  Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations.<n>This is called the Rashomon effect of (explainable) machine learning, and it raises the question of which explanations, if any, are trustworthy.<n>We propose a framework based on metamorphic testing that assesses explanation faithfulness without requiring ground-truth labels.
  arXiv  Detail & Related papers  (2026-06-04T11:57:26Z)
• Semantic Item Graph Enhancement for Multimodal Recommendation [49.66272783945571]
  Multimodal recommendation systems have attracted increasing attention for their improved performance by leveraging items' multimodal information.<n>Prior methods often build modality-specific item-item semantic graphs from raw modality features.<n>These semantic graphs suffer from semantic deficiencies, including insufficient modeling of collaborative signals among items.
  arXiv  Detail & Related papers  (2025-08-08T09:20:50Z)
• From Visual Explanations to Counterfactual Explanations with Latent Diffusion [11.433402357922414]
  We propose a new approach to tackle two key challenges in recent prominent works.<n>First, we determine which specific counterfactual features are crucial for distinguishing the "concept" of the target class from the original class.<n>Second, we provide valuable explanations for the non-robust classifier without relying on the support of an adversarially robust model.
  arXiv  Detail & Related papers  (2025-04-12T13:04:00Z)
• Plausible Extractive Rationalization through Semi-Supervised Entailment Signal [29.67884478799914]
  We take a semi-supervised approach to optimize for the plausibility of extracted rationales. We adopt a pre-trained natural language inference (NLI) model and further fine-tune it on a small set of supervised rationales. We show that, by enforcing the alignment agreement between the explanation and answer in a question-answering task, the performance can be improved without access to ground truth labels.
  arXiv  Detail & Related papers  (2024-02-13T14:12:32Z)
• Causal Generative Explainers using Counterfactual Inference: A Case Study on the Morpho-MNIST Dataset [5.458813674116228]
  We present a generative counterfactual inference approach to study the influence of visual features as well as causal factors. We employ visual explanation methods from OmnixAI open source toolkit to compare them with our proposed methods. This finding suggests that our methods are well-suited for generating highly interpretable counterfactual explanations on causal datasets.
  arXiv  Detail & Related papers  (2024-01-21T04:07:48Z)
• What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary [68.77983831618685]
  We propose to interpret the vector representations produced by dual encoders by projecting them into the model's vocabulary space. We show that the resulting projections contain rich semantic information, and draw connection between them and sparse retrieval.
  arXiv  Detail & Related papers  (2022-12-20T16:03:25Z)
• ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Model [18.537838366377915]
  ProtoVAE is a variational autoencoder-based framework that learns class-specific prototypes in an end-to-end manner. It enforces trustworthiness and diversity by regularizing the representation space and introducing an orthonormality constraint.
  arXiv  Detail & Related papers  (2022-10-15T00:42:13Z)
• Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation [89.38054401427173]
  Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment polarity of a sentence toward a specific aspect. dependency trees can be integrated into deep learning models to produce the state-of-the-art performance for ABSA. We propose a novel graph-based deep learning model to overcome these two issues.
  arXiv  Detail & Related papers  (2020-10-26T07:36:24Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)
• Understanding Neural Abstractive Summarization Models via Uncertainty [54.37665950633147]
  seq2seq abstractive summarization models generate text in a free-form manner. We study the entropy, or uncertainty, of the model's token-level predictions. We show that uncertainty is a useful perspective for analyzing summarization and text generation models more broadly.
  arXiv  Detail & Related papers  (2020-10-15T16:57:27Z)
• Explain and Improve: LRP-Inference Fine-Tuning for Image Captioning Models [82.3793660091354]
  This paper analyzes the predictions of image captioning models with attention mechanisms beyond visualizing the attention itself. We develop variants of layer-wise relevance propagation (LRP) and gradient-based explanation methods, tailored to image captioning models with attention mechanisms.
  arXiv  Detail & Related papers  (2020-01-04T05:15:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.