LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Training

• URL: http://arxiv.org/abs/2605.29888v1
• Date: Thu, 28 May 2026 13:13:49 GMT
• Title: LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Training
• Abstract summary: We propose LaRA, a layer-wise representation analysis framework for detecting contamination in large language models (LLMs)<n>We find that contamination produces progressive geometric deviations across layers, including amplified perturbation sensitivity, stronger directional collapse, and enhanced local rigidity.<n>We also develop a contamination detection protocol that aggregates representation-level deviations across layers and metrics.
• Score: 24.772253413927697
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) post-training has shown to improve reasoning in large language models (LLMs). However, there has been little exploration on the problem of data contamination in RL post-training, potentially undermining generalization and evaluation reliability of the training process itself. Existing detection methods primarily rely on output-level signals such as likelihood or entropy, which become unreliable for RL-trained models since RL shapes behavior through trajectory-level rewards rather than token likelihoods. We propose LaRA, a layer-wise representation analysis framework for detecting contamination in RL post-trained LLMs. LaRA introduces three complementary metrics, measuring perturbation sensitivity, directional collapse, and local representation rigidity under controlled perturbations. We find that contamination produces progressive geometric deviations across layers, including amplified perturbation sensitivity, stronger directional collapse, and enhanced local rigidity. Based on our findings, we also develop a contamination detection protocol that aggregates representation-level deviations across layers and metrics. Experiments on RL-trained reasoning models show that our protocol outperforms existing output-level baselines for contamination detection.

Related papers

• Robustness of Anomaly Detection Models for Industrial Control Systems under Training-Time Data Contamination [3.7391021139771325]
  This paper evaluates the robustness of offline ICS anomaly-detection pipelines on the Secure Water Treatment benchmark under training-time contamination.<n>We assess 11 heterogeneous anomaly detectors under three contamination strategies: random injection, similarity-targeted injection, and feature-noise injection.<n>The results show that robustness is strongly model-dependent and cannot be predicted from clean-data performance alone.
  arXiv  Detail & Related papers  (2026-08-24T17:49:36Z)
• SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models [51.99912169291891]
  Reinforcement learning (RL) has emerged as a promising paradigm for training reasoning-oriented models.<n>We propose a training paradigm that interleaves conventional RL with inverse reinforcement learning (IRL)<n>Our findings suggest that alternating between RL and IRL offers an effective pathway toward extending the exploration capacity of reasoning-oriented large language models.
  arXiv  Detail & Related papers  (2026-04-18T13:49:47Z)
• Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning [88.42566960813438]
  CalibRL is a hybrid-policy RLVR framework that supports controllable exploration with expert guidance.<n>CalibRL increases policy entropy in a guided manner and clarifies the target distribution.<n>Experiments across eight benchmarks, including both in-domain and out-of-domain settings, demonstrate consistent improvements.
  arXiv  Detail & Related papers  (2026-02-22T07:23:36Z)
• Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains [13.626335241662977]
  Reinforcement Learning with Verifiable Rewards (RLVR)-based post-training of Large Language Models (LLMs) has been shown to improve accuracy on reasoning tasks.<n>We investigate the effect of RL post-training on intermediate tokens which are not directly incentivized.
  arXiv  Detail & Related papers  (2025-10-20T23:58:31Z)
• Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models [30.267708813420587]
  Data contamination poses a significant threat to the reliable evaluation of Large Language Models (LLMs)<n>This issue arises when benchmark samples may inadvertently appear in training sets, compromising the validity of reported performance.<n>We propose Self-Critique as a specialized contamination detection method for RL post-training.
  arXiv  Detail & Related papers  (2025-10-10T10:58:50Z)
• On The Fragility of Benchmark Contamination Detection in Reasoning Models [20.455365567122985]
  Leaderboards for LRMs have turned evaluation into a competition, incentivizing developers to optimize directly on benchmark suites.<n>A shortcut to achieving higher rankings is to incorporate evaluation benchmarks into the training data, thereby yielding inflated performance, known as benchmark contamination.<n>We find that evading contamination detections for LRMs is alarmingly easy.
  arXiv  Detail & Related papers  (2025-09-30T21:40:54Z)
• Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning [93.00629872970364]
  Reinforcement learning (RL) has become the dominant paradigm for improving the performance of language models on complex reasoning tasks.<n>We introduce SPARKLE, a fine-grained analytic framework to dissect the effects of RL across three key dimensions.<n>We study whether difficult problems -- those yielding no RL signals and mixed-quality reasoning traces -- can still be effectively used for training.
  arXiv  Detail & Related papers  (2025-06-05T07:53:59Z)
• Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signals [49.17123504516502]
  CurrentReinforcement Fine-tuning (RFT) paradigms for Large Language Models (LLMs) suffer from inefficiency due to redundant exposure of identical queries under uniform data sampling.<n>We propose a Gradient-driven Angle-Informed Navigated RL framework.<n>By leveraging the model's intrinsic angle concentration signal, GAIN-RL dynamically selects training data in each epoch, ensuring consistently impactful gradient updates.
  arXiv  Detail & Related papers  (2025-06-02T21:40:38Z)
• RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning [125.96848846966087]
  Training large language models (LLMs) as interactive agents presents unique challenges.<n>While reinforcement learning has enabled progress in static tasks, multi-turn agent RL training remains underexplored.<n>We propose StarPO, a general framework for trajectory-level agent RL, and introduce RAGEN, a modular system for training and evaluating LLM agents.
  arXiv  Detail & Related papers  (2025-04-24T17:57:08Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL [11.058960131490903]
  Out-of-distribution (OOD) detection is a well-studied topic in supervised learning. We propose a novel task: that of Out-of-Task Distribution (OOTD) detection. We name our method GalilAI, in honor of Galileo Galilei, as it discovers, among other causal processes, that gravitational acceleration is independent of the mass of a body.
  arXiv  Detail & Related papers  (2021-10-29T01:45:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.