On the Identifiability of Masked Prediction: Mode Blindness and Mask Schedules

• URL: http://arxiv.org/abs/2608.01383v2
• Date: Fri, 07 Aug 2026 17:01:51 GMT
• Title: On the Identifiability of Masked Prediction: Mode Blindness and Mask Schedules
• Abstract summary: Masked prediction learns representations by fitting a schedule-weighted family of conditional laws.<n>It remains unclear when near-optimal conditional prediction pins down the underlying joint law.
• Score: 13.711473184679832
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Masked prediction learns representations by fitting a schedule-weighted family of conditional laws, but it remains unclear when near-optimal conditional prediction pins down the underlying joint law. We study this question for data with two well-separated global modes, outside the reach of rapid-mixing recovery guarantees, and show that the answer is decided by the mask schedule alone. Under large-context mode pinning, reweighting the two modes can move the joint law by a constant in total variation while perturbing the masked objective exponentially little in the visible-context size: mask schedules dominated by large contexts are provably blind to the global mode weights. To quantify this, we introduce an $\varepsilon$-identifiability modulus, the largest distributional error consistent with a given excess risk, and prove that it remains macroscopic at an excess risk that is exponentially small. An exact information decomposition pinpoints what restores identifiability: mode-weight sensitivity is governed by the residual mode uncertainty given the visible context. Consequently, low-visibility masks recover this sensitivity, and positive full-mask mass anchors the joint law over all admissible models with no assumption on the data law. Empirically, we test our theory at three levels: enumeration on computable laws verifies the predicted rates, gradient training reproduces both the mode blindness and the recovery, and measurements on real corpora place natural text between the two certified regimes.

Related papers

• Spatial Attention Noise Masking for Causally Sufficient Interpretability [0.21665689529884694]
  We present a novel approach to interpretability for computer vision models that dynamically masks the input image prior to classification.<n>Most interpretability methods are applied passively to already trained models, which typically result in correlational rather than causal explanations.<n>This work proposes a spatial attention noise masking framework that provides causal explanations about the features sufficient for the prediction.
  arXiv  Detail & Related papers  (2026-08-12T13:47:30Z)
• CANN-EUCLID: unsupervised constitutive artificial neural network model discovery from full-field data [0.0]
  Constitutive artificial neural networks (CANNs) provide interpretable material model discovery.<n>CANNs have so far been used in stress-inducing settings based on apparent stress-strain data from homogeneous tests.<n>Here, we combine CANNs with the stress-unsupervised full-field discovery framework EUCLID to identify sparse hyperelastic laws.
  arXiv  Detail & Related papers  (2026-06-12T15:44:06Z)
• Adaptive Order Policies for Masked Diffusion [56.27207032553083]
  We propose a scheme for learning the unmasking order using an additional lightweight policy network on top of a diffusion model.<n>Our proposed loss reweights terms in the masked diffusion loss according to policy probabilities, and results in a policy that prefers positions where the denoiser is more likely to be correct.
  arXiv  Detail & Related papers  (2026-05-29T19:26:53Z)
• Observation-Aligned Mask Priors for Learning Physical Dynamics from Authentic Occlusions [5.712248310668237]
  Observation-Aligned Mask Priors is a framework that learns the distribution of authentic observation masks.<n>We show that this intersection-based partitioning gives every valid dimension a strictly positive probability of being queried.
  arXiv  Detail & Related papers  (2026-05-16T05:23:49Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective [16.942478643768144]
  Masked Diffusion Models (MDMs) significantly accelerate inference by trading off sequential determinism.<n>We provide a unified information-theoretic framework to decouple and analyze two fundamental sources of failure: order sensitivity and parallelization bias.
  arXiv  Detail & Related papers  (2026-01-30T20:15:18Z)
• Diffusion-Guided Mask-Consistent Paired Mixing for Endoscopic Image Segmentation [57.37991748282666]
  We propose a paired, diffusion-guided paradigm that fuses the strengths of sample mixing and diffusion synthesis.<n>For each real image, a synthetic counterpart is generated under the same mask and the pair is used as a controllable input for Mask-Consistent Paired Mixing (MCPMix)<n>This produces a continuous family of intermediate samples that smoothly bridges synthetic and real appearances under shared geometry.
  arXiv  Detail & Related papers  (2025-11-05T06:14:19Z)
• Prediction with Incomplete Data under Agnostic Mask Distribution Shift [35.86200694774949]
  We consider prediction with incomplete data in the presence of distribution shift. We leverage the observation that for each mask, there is an invariant optimal predictor. We propose a novel prediction method called StableMiss.
  arXiv  Detail & Related papers  (2023-05-18T14:06:06Z)
• Uncertain Facial Expression Recognition via Multi-task Assisted Correction [43.02119884581332]
  We propose a novel method of multi-task assisted correction in addressing uncertain facial expression recognition called MTAC. Specifically, a confidence estimation block and a weighted regularization module are applied to highlight solid samples and suppress uncertain samples in every batch. Experiments on RAF-DB, AffectNet, and AffWild2 datasets demonstrate that the MTAC obtains substantial improvements over baselines when facing synthetic and real uncertainties.
  arXiv  Detail & Related papers  (2022-12-14T10:28:08Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• Asymmetric Modality Translation For Face Presentation Attack Detection [55.09300842243827]
  Face presentation attack detection (PAD) is an essential measure to protect face recognition systems from being spoofed by malicious users. We propose a novel framework based on asymmetric modality translation forPAD in bi-modality scenarios. Our method achieves state-of-the-art performance under different evaluation protocols.
  arXiv  Detail & Related papers  (2021-10-18T08:59:09Z)
• Where and What? Examining Interpretable Disentangled Representations [96.32813624341833]
  Capturing interpretable variations has long been one of the goals in disentanglement learning. Unlike the independence assumption, interpretability has rarely been exploited to encourage disentanglement in the unsupervised setting. In this paper, we examine the interpretability of disentangled representations by investigating two questions: where to be interpreted and what to be interpreted.
  arXiv  Detail & Related papers  (2021-04-07T11:22:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.