Interaction-Aware Influence Functions for Group Attribution

• URL: http://arxiv.org/abs/2605.15675v1
• Date: Fri, 15 May 2026 06:54:15 GMT
• Title: Interaction-Aware Influence Functions for Group Attribution
• Abstract summary: Influence functions approximate how removing a training example changes a quantity of interest, called the target function, such as a held-out loss.<n>We propose an interaction-aware influence function that characterizes how interactions between examples influence the target.
• Score: 26.126999503061345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Influence functions approximate how removing a training example changes a quantity of interest, called the target function, such as a held-out loss. To estimate the influence of a group of examples, the standard practice is to sum the individual influences of its members. However, this sum does not capture how examples jointly affect the target: a pair of examples may be redundant or complementary, but the sum cannot distinguish these cases. We propose an interaction-aware influence function that characterizes how interactions between examples influence the target. By expanding the target to second order around the trained parameters, we obtain an estimator that augments the standard sum with a pairwise interaction term that captures the alignment between two examples' effects on the target. We empirically evaluate our estimator in two settings. First, on six dataset-model pairs spanning logistic regression, MLPs, and ResNet-9, our estimator tracks leave-group-out retraining substantially better than first-order influence across all settings. Second, when used as a greedy selection rule for instruction-tuning data on Llama-3.1-8B, it beats prior influence-based and representation-similarity baselines on five of seven downstream tasks, in a regime where standard influence-based selection underperforms random selection.

Related papers

• Targeted Fine-Tuning of DNN-Based Receivers via Influence Functions [7.18961038438762]
  We present the first use of influence functions for deep learning-based wireless receivers.<n>We show that loss-relative influence with capacity-like binary cross-entropy loss and first-order updates on beneficial samples most consistently improves bit error rate toward genie-aided performance.
  arXiv  Detail & Related papers  (2025-09-19T13:01:30Z)
• Group-Level Data Selection for Efficient Pretraining [49.18903821780051]
  Group-MATES is an efficient group-level data selection approach to optimize the speed-quality frontier of language model pretraining.<n>Group-MATES parameterizes costly group-level selection with a relational data influence model.
  arXiv  Detail & Related papers  (2025-02-20T16:34:46Z)
• InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks [10.124267937114611]
  In this paper, we introduce InfFeed, which uses influence functions to compute the influential instances for a target instance. In doing this, InfFeed outperforms the state-of-the-art baselines by a maximum macro F1-score margin of almost 4% for hate speech classification. We also show that manually re-annotating only those silver annotated data points in the extension set that have a negative influence can immensely improve the model performance.
  arXiv  Detail & Related papers  (2024-02-22T16:59:09Z)
• If Influence Functions are the Answer, Then What is the Question? [7.873458431535409]
  Influence functions efficiently estimate the effect of removing a single training data point on a model's learned parameters. While influence estimates align well with leave-one-out retraining for linear models, recent works have shown this alignment is often poor in neural networks.
  arXiv  Detail & Related papers  (2022-09-12T16:17:43Z)
• Understanding Programmatic Weak Supervision via Source-aware Influence Function [76.74549130841383]
  Programmatic Weak Supervision (PWS) aggregates the source votes of multiple weak supervision sources into probabilistic training labels. We build on Influence Function (IF) to decompose the end model's training objective and then calculate the influence associated with each (data, source, class) These primitive influence score can then be used to estimate the influence of individual component PWS, such as source vote, supervision source, and training data.
  arXiv  Detail & Related papers  (2022-05-25T15:57:24Z)
• Rethinking InfoNCE: How Many Negative Samples Do You Need? [54.146208195806636]
  We study how many negative samples are optimal for InfoNCE in different scenarios via a semi-quantitative theoretical framework. We estimate the optimal negative sampling ratio using the $K$ value that maximizes the training effectiveness function.
  arXiv  Detail & Related papers  (2021-05-27T08:38:29Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging [112.19994766375231]
  Influence functions approximate the 'influences' of training data-points for test predictions. We present FastIF, a set of simple modifications to influence functions that significantly improves their run-time. Our experiments demonstrate the potential of influence functions in model interpretation and correcting model errors.
  arXiv  Detail & Related papers  (2020-12-31T18:02:34Z)
• Understanding Adversarial Examples from the Mutual Influence of Images and Perturbations [83.60161052867534]
  We analyze adversarial examples by disentangling the clean images and adversarial perturbations, and analyze their influence on each other. Our results suggest a new perspective towards the relationship between images and universal perturbations. We are the first to achieve the challenging task of a targeted universal attack without utilizing original training data.
  arXiv  Detail & Related papers  (2020-07-13T05:00:09Z)
• RelatIF: Identifying Explanatory Training Examples via Relative Influence [13.87851325824883]
  We use influence functions to identify relevant training examples that one might hope "explain" the predictions of a machine learning model. We introduce RelatIF, a new class of criteria for choosing relevant training examples by way of an optimization objective that places a constraint on global influence. In empirical evaluations, we find that the examples returned by RelatIF are more intuitive when compared to those found using influence functions.
  arXiv  Detail & Related papers  (2020-03-25T20:59:54Z)
• Improving Multi-Turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting [84.9716460244444]
  We consider utilizing the underlying correlation in the data resource itself to derive different kinds of supervision signals. We conduct extensive experiments in two public datasets and obtain significant improvement in both datasets.
  arXiv  Detail & Related papers  (2020-02-18T06:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.