Influence Functions for Sequence Tagging Models
- URL: http://arxiv.org/abs/2210.14177v1
- Date: Tue, 25 Oct 2022 17:13:11 GMT
- Title: Influence Functions for Sequence Tagging Models
- Authors: Sarthak Jain, Varun Manjunatha, Byron C. Wallace, Ani Nenkova
- Abstract summary: We extend influence functions to trace predictions back to the training points that informed them.
We show the practical utility of segment influence by using the method to identify systematic annotation errors.
- Score: 49.81774968547377
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Many language tasks (e.g., Named Entity Recognition, Part-of-Speech tagging,
and Semantic Role Labeling) are naturally framed as sequence tagging problems.
However, there has been comparatively little work on interpretability methods
for sequence tagging models. In this paper, we extend influence functions -
which aim to trace predictions back to the training points that informed them -
to sequence tagging tasks. We define the influence of a training instance
segment as the effect that perturbing the labels within this segment has on a
test segment level prediction. We provide an efficient approximation to compute
this, and show that it tracks with the true segment influence, measured
empirically. We show the practical utility of segment influence by using the
method to identify systematic annotation errors in two named entity recognition
corpora. Code to reproduce our results is available at
https://github.com/successar/Segment_Influence_Functions.
Related papers
- Learning to Detour: Shortcut Mitigating Augmentation for Weakly Supervised Semantic Segmentation [7.5856806269316825]
Weakly supervised semantic segmentation (WSSS) employing weak forms of labels has been actively studied to alleviate the annotation cost of acquiring pixel-level labels.
We propose shortcut mitigating augmentation (SMA) for WSSS, which generates synthetic representations of object-background combinations not seen in the training data to reduce the use of shortcut features.
arXiv Detail & Related papers (2024-05-28T13:07:35Z) - Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised
Semantic Segmentation with Its Class Label [16.745019028033518]
We propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision.
Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method.
arXiv Detail & Related papers (2024-02-27T14:51:56Z) - Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
In this paper, contextual hints are exploited via learning a context-aware classifier.
Our method is model-agnostic and can be easily applied to generic segmentation models.
With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
arXiv Detail & Related papers (2023-03-21T07:00:35Z) - Adaptive Early-Learning Correction for Segmentation from Noisy
Annotations [13.962891776039369]
We study the learning dynamics of deep segmentation networks trained on inaccurately-annotated data.
We propose a new method for segmentation from noisy annotations with two key elements.
arXiv Detail & Related papers (2021-10-07T18:46:23Z) - Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised
Semantic Segmentation [88.49669148290306]
We propose a novel weakly supervised multi-task framework called AuxSegNet to leverage saliency detection and multi-label image classification as auxiliary tasks.
Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations.
The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks.
arXiv Detail & Related papers (2021-07-25T11:39:58Z) - Dynamic Semantic Matching and Aggregation Network for Few-shot Intent
Detection [69.2370349274216]
Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances.
Semantic components are distilled from utterances via multi-head self-attention.
Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
arXiv Detail & Related papers (2020-10-06T05:16:38Z) - Weakly Supervised Temporal Action Localization with Segment-Level Labels [140.68096218667162]
Temporal action localization presents a trade-off between test performance and annotation-time cost.
We introduce a new segment-level supervision setting: segments are labeled when annotators observe actions happening here.
We devise a partial segment loss regarded as a loss sampling to learn integral action parts from labeled segments.
arXiv Detail & Related papers (2020-07-03T10:32:19Z) - Explaining Black Box Predictions and Unveiling Data Artifacts through
Influence Functions [55.660255727031725]
Influence functions explain the decisions of a model by identifying influential training examples.
We conduct a comparison between influence functions and common word-saliency methods on representative tasks.
We develop a new measure based on influence functions that can reveal artifacts in training data.
arXiv Detail & Related papers (2020-05-14T00:45:23Z) - Lexical Semantic Recognition [15.944476972081345]
We train a neural CRF sequence tagger and evaluate its performance along various axes of annotation.
Our work also establishes baseline models and evaluation metrics for integrated and accurate modeling of lexical semantics.
arXiv Detail & Related papers (2020-04-30T17:52:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.