Refining the Information Bottleneck via Adversarial Information Separation

• URL: http://arxiv.org/abs/2602.06549v2
• Date: Mon, 09 Feb 2026 02:08:49 GMT
• Title: Refining the Information Bottleneck via Adversarial Information Separation
• Authors: Shuai Ning, Zhenpeng Wang, Lin Wang, Bing Chen, Shuangrong Liu, Xu Wu, Jin Zhou, Bo Yang,
• Abstract summary: Generalizing from limited data is critical for models in domains such as material science.<n>We propose the Adversarial Information Separation Framework (AdverISF), which isolates task-relevant features from noise.<n>We show that AdverISF outperforms state-of-the-art methods in data-scarce scenarios.
• Score: 10.748014850495144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalizing from limited data is particularly critical for models in domains such as material science, where task-relevant features in experimental datasets are often heavily confounded by measurement noise and experimental artifacts. Standard regularization techniques fail to precisely separate meaningful features from noise, while existing adversarial adaptation methods are limited by their reliance on explicit separation labels. To address this challenge, we propose the Adversarial Information Separation Framework (AdverISF), which isolates task-relevant features from noise without requiring explicit supervision. AdverISF introduces a self-supervised adversarial mechanism to enforce statistical independence between task-relevant features and noise representations. It further employs a multi-layer separation architecture that progressively recycles noise information across feature hierarchies to recover features inadvertently discarded as noise, thereby enabling finer-grained feature extraction. Extensive experiments demonstrate that AdverISF outperforms state-of-the-art methods in data-scarce scenarios. In addition, evaluations on real-world material design tasks show that it achieves superior generalization performance.

Related papers

• Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise [51.179816451161635]
  Diffusion models have achieved remarkable progress in image generation, but their increasing deployment raises serious concerns about privacy.<n>In this work, we utilize a critical yet overlooked vulnerability: the widely used noise schedules fail to fully eliminate semantic information in the images.<n>We propose a simple yet effective membership inference attack, which injects semantic information into the initial noise and infers membership by analyzing the model's generation result.
  arXiv  Detail & Related papers  (2026-01-29T12:29:01Z)
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• FAIM: Frequency-Aware Interactive Mamba for Time Series Classification [87.84511960413715]
  Time series classification (TSC) is crucial in numerous real-world applications, such as environmental monitoring, medical diagnosis, and posture recognition.<n>We propose FAIM, a lightweight Frequency-Aware Interactive Mamba model.<n>We show that FAIM consistently outperforms existing state-of-the-art (SOTA) methods, achieving a superior trade-off between accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-11-26T08:36:33Z)
• FINDER: Feature Inference on Noisy Datasets using Eigenspace Residuals [0.7349919073512893]
  We introduce FINDER, a rigorous framework for analyzing generic classification problems.<n>FINDER incorporates fundamental analysis ideas into the feature learning and inference stages.<n>We produce state of the art breakthroughs in: (i) Alzheimer's Disease stage classification, (ii) Remote sensing detection of deforestation.
  arXiv  Detail & Related papers  (2025-10-22T18:00:03Z)
• Collaborative Learning of Scattering and Deep Features for SAR Target Recognition with Noisy Labels [7.324728751991982]
  We propose collaborative learning of scattering and deep features (DF) for SAR automatic target recognition with noisy labels.<n>Specifically, a multi-model feature fusion framework is designed to integrate scattering and deep features.<n>The proposed method can achieve state-of-the-art performance under different operating conditions with various label noises.
  arXiv  Detail & Related papers  (2025-08-11T06:10:23Z)
• Offline Action-Free Learning of Ex-BMDPs by Comparing Diverse Datasets [87.62730694973696]
  This paper introduces CRAFT, a sample-efficient algorithm leveraging differences in controllable feature dynamics across agents to learn representations.<n>We provide theoretical guarantees for CRAFT's performance and demonstrate its feasibility on a toy example.
  arXiv  Detail & Related papers  (2025-03-26T22:05:57Z)
• RobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Response [23.45168175163634]
  Supervised fine-tuning (SFT) plays a crucial role in adapting large language models (LLMs) to specific domains or tasks.<n>We introduce a robust SFT framework (RobustFT) that performs noise detection and relabeling on downstream task data.<n>Extensive experiments conducted on multiple LLMs across five datasets demonstrate RobustFT's exceptional performance in noisy scenarios.
  arXiv  Detail & Related papers  (2024-12-19T15:00:18Z)
• LoSAM: Local Search in Additive Noise Models with Mixed Mechanisms and General Noise for Global Causal Discovery [2.4305626489408465]
  Local search in additive noise models, LoSAM, is a topological ordering for learning a unique DAG in ANMs with mixed causal mechanisms and general noise distributions.<n>We prove consistency and criteria, ensuring scalability and sample efficiency.
  arXiv  Detail & Related papers  (2024-10-15T16:28:55Z)
• Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances [55.37242480995541]
  We propose to denoise noisy NER data with guidance from a small set of clean instances. Along with the main NER model we train a discriminator model and use its outputs to recalibrate the sample weights. Results on public crowdsourcing and distant supervision datasets show that the proposed method can consistently improve performance with a small guidance set.
  arXiv  Detail & Related papers  (2023-10-25T17:23:37Z)
• Learning Confident Classifiers in the Presence of Label Noise [5.551384206194696]
  This paper proposes a probabilistic model for noisy observations that allows us to build a confident classification and segmentation models.<n>Our experiments show that our algorithm outperforms state-of-the-art solutions for the considered classification and segmentation problems.
  arXiv  Detail & Related papers  (2023-01-02T04:27:25Z)
• Representation Learning for the Automatic Indexing of Sound Effects Libraries [79.68916470119743]
  We show that a task-specific but dataset-independent representation can successfully address data issues such as class imbalance, inconsistent class labels, and insufficient dataset size. Detailed experimental results show the impact of metric learning approaches and different cross-dataset training methods on representational effectiveness.
  arXiv  Detail & Related papers  (2022-08-18T23:46:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.