Disagreement-Regularized Importance Sampling for Adversarial Label Corruption

• URL: http://arxiv.org/abs/2605.07551v1
• Date: Fri, 08 May 2026 10:27:51 GMT
• Title: Disagreement-Regularized Importance Sampling for Adversarial Label Corruption
• Abstract summary: Disagreement-Regularized Importance Sampling (DR-IS) is a sub-sampling method based on loss rank-disagreement across independent proxy ensemble.<n>DR-IS is robust under targeted high-norm attacks that break magnitude-based methods such as the Error $L$-norm (EL2N) on benchmark datasets.
• Score: 3.0416928830191066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Standard Importance Sampling (IS) collapses under label corruption because high-norm examples, prioritized for variance reduction, are often adversarial outliers. We formalize this misalignment using an $\varepsilon$-contamination model and propose Disagreement-Regularized Importance Sampling (DR-IS), a sub-sampling method based on loss rank-disagreement across independent proxy ensemble. We prove finite-sample concentration bounds showing that the empirical rank disagreement of bulk corrupted examples is bounded above, and that of boundary-clean examples bounded below, both at rate $O(\sqrt{\log(N/δ)/K})$ with probability $1-δ$; when the structural expectation gap $Δ'$ between the two groups is positive and the boundary-clean set is at least as large as the selected subset, these bounds certify strict separation and control the contamination rate of the selected subset. Empirically, DR-IS remains robust under targeted high-norm attacks that break magnitude-based methods such as the Error $L_2$-norm (EL2N) on benchmark datasets. DR-IS complements training-dynamics approaches like Area Under the Margin ranking (AUM), offering improved robustness in the loss-aligned regime alongside explicit finite-sample concentration certificates and a contamination bound limiting noise leakage from the statistical tail of corrupted points.

Related papers

• Conformal-DRO: Distributionally Robust Optimization with Conformalized Ambiguity Set [9.492534748349316]
  This paper studies latent distributional heterogeneity, where each instance has an unobserved law but contributes only one observation.<n>We propose Conformal-DRO, which uses nested conformal regions to construct an ambiguity set for the future latent law.<n>The resulting robust value provides a finite-sample certificate for the selected decision's expected cost.
  arXiv  Detail & Related papers  (2026-09-10T04:30:21Z)
• How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? [58.740078141879984]
  Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on.<n>Recent work in causality-based domain generalization has shown how shared causal structure can induce invariant predictors.<n>In this paper, we study when full or partial causal knowledge can provably improve supervised domain adaptation.
  arXiv  Detail & Related papers  (2026-06-10T21:07:49Z)
• Anytime PAC-Bayes for Constrained Density-Ratio Networks under Covariate Shift [0.0]
  PAC-Bayes is instantiated on the weighted risk in a fixed-time regime that yields Bernoulli-KL bounds.<n>A patch test against analytic ground truth with a real-data deployment validates the framework.<n>A single fixed-time coverage failure is recorded, with per-split coverage aligning one-to-one with the magnitude of the label shift.
  arXiv  Detail & Related papers  (2026-05-17T01:07:17Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Distributionally Robust K-Means Clustering [18.037323759791658]
  K-means clustering is notoriously brittle to outliers, distribution shifts, and limited sample sizes.<n>We develop a distributionally robust variant that protects against such pathologies.<n>A tractable dual yields a soft-clustering scheme that replaces hard assignments with smoothly weighted ones.
  arXiv  Detail & Related papers  (2026-04-13T07:32:36Z)
• Non-monotonicity in Conformal Risk Control [4.71350885287928]
  Conformal risk control (CRC) provides distribution-free guarantees for controlling the expected loss at a user-specified level.<n>We study CRC under non-monotone loss functions when the tuning parameter is selected from a finite grid.<n>We show that the validity of CRC without monotonicity depends on the relationship between the calibration sample size and the grid resolution.
  arXiv  Detail & Related papers  (2026-04-02T00:26:50Z)
• Fair regression under localized demographic parity constraints [9.082079838093371]
  We introduce a novel ($ell$, Z)-fair predictor, which imposes groupwise CDF constraints of the form F f |S=s (z m ) = $ell$ m for prescribed pairs.<n> Experiments on synthetic and real datasets illustrate an interpretable fairness-accuracy trade-off, enabling targeted corrections at decision-relevant quantiles or thresholds.
  arXiv  Detail & Related papers  (2026-03-26T09:21:05Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions [0.0]
  In genomics experiments, exchangeability often holds only within subsets of interventions that leave a target variable "unaffected"<n>Our contributions are: (i) a contamination-robust conformal coverage theorem that quantifies how misclassification of "unaffected" calibration examples degrades coverage via an explicit function $g(,n)$ of the contamination fraction and calibration set size; and (ii) a task-driven partial causal learning formulation that estimates only the binary descendant indicators $Z_a,i=mathbf1iinmathrmdesc(a)$ needed
  arXiv  Detail & Related papers  (2026-03-02T18:58:22Z)
• Bulk-Calibrated Credal Ambiguity Sets: Fast, Tractable Decision Making under Out-of-Sample Contamination [8.826173150779145]
  Distributionally robust optimisation (DRO) minimises the worst-case expected loss over an ambiguity set.<n>We show how IP credal sets translate into DRO objectives with interpretable tolerance levels.
  arXiv  Detail & Related papers  (2026-01-29T06:37:36Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling [77.15766509677348]
  Conditional generative models often inherit spurious correlations from the training dataset. This can result in label-conditional distributions that are imbalanced with respect to another latent attribute. We propose a general two-step strategy to mitigate this issue.
  arXiv  Detail & Related papers  (2022-12-05T08:09:33Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity [55.29408396918968]
  We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification. Our contributions include both consistency and robustness by establishing top-$k$ consistency of LDR losses for multi-class classification. We propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance.
  arXiv  Detail & Related papers  (2021-12-30T00:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.