Language models recognize dropout and Gaussian noise applied to their activations

• URL: http://arxiv.org/abs/2604.17465v1
• Date: Sun, 19 Apr 2026 14:30:13 GMT
• Title: Language models recognize dropout and Gaussian noise applied to their activations
• Abstract summary: We verbalize either (a) emphmask activations, simulating emphdropout, or (b) add emphGaussian noise to them.<n>We test models from the Llama, Olmo, and Qwen families, with sizes between 8B and 32B.
• Score: 74.06294367754748
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We provide evidence that language models can detect, localize and, to a certain degree, verbalize the difference between perturbations applied to their activations. More precisely, we either (a) \emph{mask} activations, simulating \emph{dropout}, or (b) add \emph{Gaussian noise} to them, at a target sentence. We then ask a multiple-choice question such as ``\emph{Which of the previous sentences was perturbed?}'' or ``\emph{Which of the two perturbations was applied?}''. We test models from the Llama, Olmo, and Qwen families, with sizes between 8B and 32B, all of which can easily detect and localize the perturbations, often with perfect accuracy. These models can also learn, when taught in context, to distinguish between dropout and Gaussian noise. Notably, \qwenb's \emph{zero-shot} accuracy in identifying which perturbation was applied improves as a function of the perturbation strength and, moreover, decreases if the in-context labels are flipped, suggesting a prior for the correct ones -- even modulo controls. Because dropout has been used as a training-regularization technique, while Gaussian noise is sometimes added during inference, we discuss the possibility of a data-agnostic ``training awareness'' signal and the implications for AI safety. The code and data are available at \href{https://github.com/saifh-github/llm-dropout-noise-recognition}{link 1} and \href{https://drive.google.com/file/d/1es-Sfw_AH9GficeXgeqpy87rocrZZ_PQ/view}{link 2}, respectively.

Related papers

• When Is Noise Response Universal? Tokenization as the Hidden Variable in Language Models [3.6243102296545846]
  We study the degradation rate across neural models, both sentence embeddings and decoder-only LLMs.<n>We find that under word-level noise, models with very different architectures decline along nearly the same curve, while under character-level noise they separate.<n>This finding and its underlying mechanism provide a practical means to predict a model's robustness to noise without any noisy evaluation.
  arXiv  Detail & Related papers  (2026-08-26T18:51:51Z)
• Does Semantic Noise Initialization Transfer from Images to Videos? A Paired Diagnostic Study [0.7166401281812406]
  temporal coupling can introduce extra degrees of freedom and instability in text-to-video (T2V) generation.<n>We benchmark semantic noise initialization against standard Gaussian noise using a frozen VideoCrafter-style T2V diffusion backbone and VBench on 100 prompts.<n>We observe a small positive trend on temporal-related dimensions; however, the 95 percent confidence interval includes zero (p 0.17) and the overall score remains on par with the baseline.
  arXiv  Detail & Related papers  (2026-03-03T07:36:07Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression [8.823417072582348]
  Supervised learning for empathy regression is challenged by noisy self-reported empathy scores.<n>We propose UPLME, an uncertainty-aware probabilistic language modelling framework to capture label noise in the regression setting.
  arXiv  Detail & Related papers  (2025-08-05T14:46:28Z)
• Understanding the Effect of Noise in LLM Training Data with Algorithmic Chains of Thought [0.0]
  We study how noise in chain of thought impacts task performance in highly-controlled setting. We define two types of noise: textitstatic noise, a local form of noise which is applied after the CoT trace is computed, and textitdynamic noise, a global form of noise which propagates errors in the trace as it is computed. We find fine-tuned models are extremely robust to high levels of static noise but struggle significantly more with lower levels of dynamic noise.
  arXiv  Detail & Related papers  (2024-02-06T13:59:56Z)
• Label Noise: Correcting the Forward-Correction [0.0]
  Training neural network classifiers on datasets with label noise poses a risk of overfitting them to the noisy labels. We propose an approach to tackling overfitting caused by label noise. Motivated by this observation, we propose imposing a lower bound on the training loss to mitigate overfitting.
  arXiv  Detail & Related papers  (2023-07-24T19:41:19Z)
• NoiseCAM: Explainable AI for the Boundary Between Noise and Adversarial Attacks [21.86821880164293]
  adversarial attacks can easily mislead a neural network and lead to wrong decisions. In this paper, we use the gradient class activation map (GradCAM) to analyze the behavior deviation of the VGG-16 network. We also propose a novel NoiseCAM algorithm that integrates information from globally and pixel-level weighted class activation maps.
  arXiv  Detail & Related papers  (2023-03-09T22:07:41Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)
• Consistency Regularization for Certified Robustness of Smoothed Classifiers [89.72878906950208]
  A recent technique of randomized smoothing has shown that the worst-case $ell$-robustness can be transformed into the average-case robustness. We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise.
  arXiv  Detail & Related papers  (2020-06-07T06:57:43Z)
• Towards Noise-resistant Object Detection with Noisy Annotations [119.63458519946691]
  Training deep object detectors requires significant amount of human-annotated images with accurate object labels and bounding box coordinates. Noisy annotations are much more easily accessible, but they could be detrimental for learning. We address the challenging problem of training object detectors with noisy annotations, where the noise contains a mixture of label noise and bounding box noise.
  arXiv  Detail & Related papers  (2020-03-03T01:32:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.