Machine Learning from Explanations

• URL: http://arxiv.org/abs/2507.04788v1
• Date: Mon, 07 Jul 2025 09:09:52 GMT
• Title: Machine Learning from Explanations
• Authors: Jiashu Tao, Reza Shokri,
• Abstract summary: We introduce an innovative approach for training reliable classification models on smaller datasets.<n>Our method centers around a two-stage training cycle that alternates between enhancing model prediction accuracy and refining its attention to match the explanations.<n>We demonstrate that our training cycle expedites the convergence towards more accurate and reliable models.
• Score: 17.28638946021444
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Acquiring and training on large-scale labeled data can be impractical due to cost constraints. Additionally, the use of small training datasets can result in considerable variability in model outcomes, overfitting, and learning of spurious correlations. A crucial shortcoming of data labels is their lack of any reasoning behind a specific label assignment, causing models to learn any arbitrary classification rule as long as it aligns data with labels. To overcome these issues, we introduce an innovative approach for training reliable classification models on smaller datasets, by using simple explanation signals such as important input features from labeled data. Our method centers around a two-stage training cycle that alternates between enhancing model prediction accuracy and refining its attention to match the explanations. This instructs models to grasp the rationale behind label assignments during their learning phase. We demonstrate that our training cycle expedites the convergence towards more accurate and reliable models, particularly for small, class-imbalanced training data, or data with spurious features.

Related papers

• Learning from Noisy Labels via Self-Taught On-the-Fly Meta Loss Rescaling [6.861041888341339]
  We propose unsupervised on-the-fly meta loss rescaling to reweight training samples.<n>We are among the first to attempt on-the-fly training data reweighting on the challenging task of dialogue modeling.<n>Our strategy is robust in the face of noisy and clean data, handles class imbalance, and prevents overfitting to noisy labels.
  arXiv  Detail & Related papers  (2024-12-17T14:37:50Z)
• For Better or For Worse? Learning Minimum Variance Features With Label Augmentation [7.183341902583164]
  In this work, we analyze the role played by the label augmentation aspect of data augmentation methods.<n>We first prove that linear models on binary classification data trained with label augmentation learn only the minimum variance features in the data.<n>We then use our techniques to show that even for nonlinear models and general data distributions, the label smoothing and Mixup losses are lower bounded by a function of the model output variance.
  arXiv  Detail & Related papers  (2024-02-10T01:36:39Z)
• Clarify: Improving Model Robustness With Natural Language Corrections [59.041682704894555]
  The standard way to teach models is by feeding them lots of data. This approach often teaches models incorrect ideas because they pick up on misleading signals in the data. We propose Clarify, a novel interface and method for interactively correcting model misconceptions.
  arXiv  Detail & Related papers  (2024-02-06T05:11:38Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification.<n>XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations.<n>Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Boosting Semi-Supervised Learning by bridging high and low-confidence predictions [4.18804572788063]
  Pseudo-labeling is a crucial technique in semi-supervised learning (SSL) We propose a new method called ReFixMatch, which aims to utilize all of the unlabeled data during training.
  arXiv  Detail & Related papers  (2023-08-15T00:27:18Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• A Benchmark Generative Probabilistic Model for Weak Supervised Learning [2.0257616108612373]
  Weak Supervised Learning approaches have been developed to alleviate the annotation burden. We show that latent variable models (PLVMs) achieve state-of-the-art performance across four datasets.
  arXiv  Detail & Related papers  (2023-03-31T07:06:24Z)
• Constructing Balance from Imbalance for Long-tailed Image Recognition [50.6210415377178]
  The imbalance between majority (head) classes and minority (tail) classes severely skews the data-driven deep neural networks. Previous methods tackle with data imbalance from the viewpoints of data distribution, feature space, and model design. We propose a concise paradigm by progressively adjusting label space and dividing the head classes and tail classes. Our proposed model also provides a feature evaluation method and paves the way for long-tailed feature learning.
  arXiv  Detail & Related papers  (2022-08-04T10:22:24Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• Machine Unlearning of Features and Labels [72.81914952849334]
  We propose first scenarios for unlearning and labels in machine learning models. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
  arXiv  Detail & Related papers  (2021-08-26T04:42:24Z)
• Learning from Noisy Labels for Entity-Centric Information Extraction [17.50856935207308]
  We propose a simple co-regularization framework for entity-centric information extraction. These models are jointly optimized with task-specific loss, and are regularized to generate similar predictions. In the end, we can take any of the trained models for inference.
  arXiv  Detail & Related papers  (2021-04-17T22:49:12Z)
• Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
  Self-Tuning is a novel approach to enable data-efficient deep learning. It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model. It outperforms its SSL and TL counterparts on five tasks by sharp margins.
  arXiv  Detail & Related papers  (2021-02-25T14:56:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.