Directional Influence Function: Estimating Training Data Influence in Constrained Learning

• URL: http://arxiv.org/abs/2607.23388v3
• Date: Wed, 29 Jul 2026 01:25:56 GMT
• Title: Directional Influence Function: Estimating Training Data Influence in Constrained Learning
• Abstract summary: Directional Influence Function (DIF) is a novel estimator that explicitly incorporates constraints into influence estimation.<n>Our results establish DIF as an efficient and reliable tool for data attribution in constrained learning.
• Score: 2.9843488044412507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As constrained learning becomes increasingly common, models are trained under explicit feasibility requirements to enforce fairness, safety, robustness, regulariza- tion, and physics or logic constraints. Understanding how training samples in- fluence the model solution (e.g., learned parameters) is crucial for interpretability and robustness. The classical influence function (IF) estimates sample contribu- tions via local sensitivity analysis, measuring how the solution changes when a specific training sample is perturbed or removed. However, IF becomes unreli- able in constrained settings: data perturbations can reshape both the objective and the feasible region, leading to estimates that violate feasibility. In response, we propose the Directional Influence Function (DIF), a novel estimator that explicitly incorporates these constraints into influence estimation. DIF formulates the opti- mality conditions of constrained learning as a variational inequality (VI) and ana- lyzes how perturbing training data affects this VI. We validate DIF on constrained linear regression and demonstrate that it recovers leave-one-out retraining results, whereas IF and penalty-based IF exhibit significant bias. We further apply DIF to fairness-constrained CNNs, where DIF accurately predicts test loss changes under data removal and aligns closely with actual retraining. Our results establish DIF as an efficient and reliable tool for data attribution in constrained learning.

Related papers

• Non-Parametric Rehearsal Learning via Conditional Mean Embeddings [88.89267783967263]
  We propose a non-parametric rehearsal learning approach for textitavoiding undesired future (AUF) problems.<n>Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes.
  arXiv  Detail & Related papers  (2026-05-09T15:30:52Z)
• FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection [63.91351553178842]
  FACTOR is a training-free Test-time adaptation for Open-vocabulary object detection.<n>It quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions.<n>Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods.
  arXiv  Detail & Related papers  (2026-05-05T02:31:18Z)
• Learning Complex Physical Regimes via Coverage-oriented Uncertainty Quantification: An application to the Critical Heat Flux [0.0]
  Uncertainty quantification (UQ) should not be viewed as a safety assessment, but as a support to the learning task itself.<n>We focus on the Critical Heat Flux benchmark and dataset presented by the OECD/NEA Expert Group on Reactor Systems Multi-Physics.<n>We show that while post-hoc methods ensure statistical calibration, coverage-oriented learning effectively reshapes the model's representation to match the complex physical regimes.
  arXiv  Detail & Related papers  (2026-02-25T09:04:15Z)
• f-INE: A Hypothesis Testing Framework for Estimating Influence under Training Randomness [13.922293647552303]
  We introduce *f-influence* -- a new influence estimation framework grounded in hypothesis testing.<n>We also design a highly efficient algorithm **f**-**IN**fluence **E**stimation (**f-INE**) that computes f-influence **in a single training run.<n>Finally, we scale up f-INE to estimate influence of instruction tuning data on Llama-3.1-8B and show it can reliably detect poisoned samples.
  arXiv  Detail & Related papers  (2025-10-12T09:05:47Z)
• Lightweight and Robust Federated Data Valuation [24.16107496848504]
  Federated learning (FL) faces persistent challenges due to non-IID data distributions and adversarial client behavior.<n>We propose FedIF, a novel FL aggregation framework that leverages trajectory-based influence estimation to efficiently compute client contributions.<n>Our results establish FedIF as a practical, theoretically grounded, and scalable alternative to Shapley-value-based approaches for efficient and robust FL in real-world deployments.
  arXiv  Detail & Related papers  (2025-09-29T22:31:56Z)
• Data Fusion for Partial Identification of Causal Effects [62.56890808004615]
  We propose a novel partial identification framework that enables researchers to answer key questions.<n>Is the causal effect positive or negative? and How severe must assumption violations be to overturn this conclusion?<n>We apply our framework to the Project STAR study, which investigates the effect of classroom size on students' third-grade standardized test performance.
  arXiv  Detail & Related papers  (2025-05-30T07:13:01Z)
• CALF: A Conditionally Adaptive Loss Function to Mitigate Class-Imbalanced Segmentation [0.2902243522110345]
  Imbalanced datasets pose a challenge in training deep learning (DL) models for medical diagnostics.<n>We propose a novel, statistically driven, conditionally adaptive loss function (CALF) tailored to accommodate the conditions of imbalanced datasets in DL training.
  arXiv  Detail & Related papers  (2025-04-06T12:03:33Z)
• Feasible Learning [78.6167929413604]
  We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample.<n>Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.
  arXiv  Detail & Related papers  (2025-01-24T20:39:38Z)
• A Versatile Influence Function for Data Attribution with Non-Decomposable Loss [3.1615846013409925]
  We propose a Versatile Influence Function (VIF) that can be straightforwardly applied to machine learning models trained with any non-decomposable loss.<n>VIF represents a significant advancement in data attribution, enabling efficient influence-function-based attribution across a wide range of machine learning paradigms.
  arXiv  Detail & Related papers  (2024-12-02T09:59:01Z)
• Estimating Structural Target Functions using Machine Learning and Influence Functions [103.47897241856603]
  We propose a new framework for statistical machine learning of target functions arising as identifiable functionals from statistical models. This framework is problem- and model-agnostic and can be used to estimate a broad variety of target parameters of interest in applied statistics. We put particular focus on so-called coarsening at random/doubly robust problems with partially unobserved information.
  arXiv  Detail & Related papers  (2020-08-14T16:48:29Z)
• Influence Functions in Deep Learning Are Fragile [52.31375893260445]
  influence functions approximate the effect of samples in test-time predictions. influence estimates are fairly accurate for shallow networks. Hessian regularization is important to get highquality influence estimates.
  arXiv  Detail & Related papers  (2020-06-25T18:25:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.