LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference

• URL: http://arxiv.org/abs/2505.22848v2
• Date: Tue, 03 Jun 2025 09:41:25 GMT
• Title: LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
• Authors: Pingjun Hong, Beiduo Chen, Siyao Peng, Marie-Catherine de Marneffe, Barbara Plank,
• Abstract summary: We introduce LITEX, a linguistically-informed taxonomy for categorizing free-text explanations.<n>Using this taxonomy, we annotate a subset of the e-SNLI dataset, validate the taxonomy's reliability, and analyze how it aligns with NLI labels, highlights, and explanations.
• Score: 23.28476268369439
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There is increasing evidence of Human Label Variation (HLV) in Natural Language Inference (NLI), where annotators assign different labels to the same premise-hypothesis pair. However, within-label variation--cases where annotators agree on the same label but provide divergent reasoning--poses an additional and mostly overlooked challenge. Several NLI datasets contain highlighted words in the NLI item as explanations, but the same spans on the NLI item can be highlighted for different reasons, as evidenced by free-text explanations, which offer a window into annotators' reasoning. To systematically understand this problem and gain insight into the rationales behind NLI labels, we introduce LITEX, a linguistically-informed taxonomy for categorizing free-text explanations. Using this taxonomy, we annotate a subset of the e-SNLI dataset, validate the taxonomy's reliability, and analyze how it aligns with NLI labels, highlights, and explanations. We further assess the taxonomy's usefulness in explanation generation, demonstrating that conditioning generation on LITEX yields explanations that are linguistically closer to human explanations than those generated using only labels or highlights. Our approach thus not only captures within-label variation but also shows how taxonomy-guided generation for reasoning can bridge the gap between human and model explanations more effectively than existing strategies.

Related papers

• Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions [50.92702206798324]
  We propose a generative prompting framework for zero-shot text classification. GEN-Z measures the LM likelihood of input text conditioned on natural language descriptions of labels. We show that zero-shot classification with simple contextualization of the data source consistently outperforms both zero-shot and few-shot baselines.
  arXiv  Detail & Related papers  (2023-11-13T07:12:57Z)
• Ecologically Valid Explanations for Label Variation in NLI [27.324994764803808]
  We build LiveNLI, an English dataset of 1,415 ecologically valid explanations (annotators explain the NLI labels they chose) for 122 MNLI items. LiveNLI explanations confirm that people can systematically vary on their interpretation and highlight within-label variation. This suggests that explanations are crucial for navigating label interpretations in general.
  arXiv  Detail & Related papers  (2023-10-20T22:52:19Z)
• HuBERTopic: Enhancing Semantic Representation of HuBERT through Self-supervision Utilizing Topic Model [62.995175485416]
  We propose a new approach to enrich the semantic representation of HuBERT. An auxiliary topic classification task is added to HuBERT by using topic labels as teachers. Experimental results demonstrate that our method achieves comparable or better performance than the baseline in most tasks.
  arXiv  Detail & Related papers  (2023-10-06T02:19:09Z)
• X-PARADE: Cross-Lingual Textual Entailment and Information Divergence across Paragraphs [55.80189506270598]
  X-PARADE is the first cross-lingual dataset of paragraph-level information divergences. Annotators label a paragraph in a target language at the span level and evaluate it with respect to a corresponding paragraph in a source language. Aligned paragraphs are sourced from Wikipedia pages in different languages.
  arXiv  Detail & Related papers  (2023-09-16T04:34:55Z)
• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis [3.515619810213763]
  We propose using automatically generated natural language definitions of contextualised word usages as interpretable word and word sense representations. We demonstrate how the resulting sense labels can make existing approaches to semantic change analysis more interpretable.
  arXiv  Detail & Related papers  (2023-05-19T20:36:21Z)
• Understanding and Predicting Human Label Variation in Natural Language Inference through Explanation [18.161206115232066]
  We create the first ecologically valid explanation dataset with diverse reasoning, LiveNLI. LiveNLI contains annotators' highlights and free-text explanations for the label(s) of their choice for 122 English Natural Language Inference items. We used its explanations for chain-of-thought prompting, and found there is still room for improvement in GPT-3's ability to predict label distribution with in-context learning.
  arXiv  Detail & Related papers  (2023-04-24T20:45:09Z)
• Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures [104.32063681736349]
  We present an approach to describe predicate-argument structures using natural language definitions instead of discrete labels. Our experiments and analyses on PropBank-style and FrameNet-style, dependency-based and span-based SRL also demonstrate that a flexible model with an interpretable output does not necessarily come at the expense of performance.
  arXiv  Detail & Related papers  (2022-12-02T11:19:16Z)
• Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization [80.94424037751243]
  In zero-shot multilingual extractive text summarization, a model is typically trained on English dataset and then applied on summarization datasets of other languages. We propose NLS (Neural Label Search for Summarization), which jointly learns hierarchical weights for different sets of labels together with our summarization model. We conduct multilingual zero-shot summarization experiments on MLSUM and WikiLingua datasets, and we achieve state-of-the-art results using both human and automatic evaluations.
  arXiv  Detail & Related papers  (2022-04-28T14:02:16Z)
• LIREx: Augmenting Language Inference with Relevant Explanation [1.4780878458667916]
  Natural language explanations (NLEs) are a form of data annotation in which annotators identify rationales when assigning labels to data instances. NLEs have been shown to capture human reasoning better, but not as beneficial for natural language inference. We propose a novel framework, LIREx, that incorporates both a rationale-enabled explanation generator and an instance selector to select only relevant NLEs.
  arXiv  Detail & Related papers  (2020-12-16T18:49:29Z)
• NILE : Natural Language Inference with Faithful Natural Language Explanations [10.074153632701952]
  We propose Natural-language Inference over Label-specific Explanations (NILE) NILE is a novel NLI method which utilizes auto-generated label-specific explanations to produce labels along with its faithful explanation. We discuss the faithfulness of NILE's explanations in terms of sensitivity of the decisions to the corresponding explanations.
  arXiv  Detail & Related papers  (2020-05-25T13:56:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.