SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning

• URL: http://arxiv.org/abs/2605.09266v2
• Date: Tue, 12 May 2026 14:15:36 GMT
• Title: SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
• Abstract summary: SeePhys Pro is a benchmark that studies whether models preserve the same reasoning capability when critical information is progressively transferred from text to image.<n>Our evaluation shows that performance degrades on average as information moves from language to diagrams, with visual variable grounding as the most critical bottleneck.<n>To analyze this effect, text-deletion, image-mask-rate, and format-saturation controls suggest that such gains can arise from residual textual and distributional cues rather than valid visual evidence.
• Score: 37.01284827211354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce SeePhys Pro, a fine-grained modality transfer benchmark that studies whether models preserve the same reasoning capability when critical information is progressively transferred from text to image. Unlike standard vision-essential benchmarks that evaluate a single input form, SeePhys Pro features four semantically aligned variants for each problem with progressively increasing visual elements. Our evaluation shows that current frontier models are far from representation-invariant reasoners: performance degrades on average as information moves from language to diagrams, with visual variable grounding as the most critical bottleneck. Motivated by this inference-time fragility, we further develop large training corpora for multimodal RLVR and use blind training as a diagnostic control, finding that RL with all training images masked can still improve performance on unmasked validation sets. To analyze this effect, text-deletion, image-mask-rate, and format-saturation controls suggest that such gains can arise from residual textual and distributional cues rather than valid visual evidence. Our results highlight the need to evaluate multimodal reasoning not only by final-answer accuracy, but also by robustness under modality transfer and by diagnostics that test whether improvements rely on task-critical visual evidence.

Related papers

• Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models [14.5102059776721]
  Multimodal Large Language Models (MLLMs) achieve strong performance by integrating visual inputs with the rich priors of pretrained language models.<n>However, they often fail on vision-centric tasks, especially when visual evidence conflicts with pretrained knowledge.<n>We explore these failures using two diagnostic paradigms: (1) probing whether visual information is available, via image reconstruction, and (2) measuring multimodal context sensitivity.
  arXiv  Detail & Related papers  (2026-07-28T22:52:34Z)
• OmniPhys: Knowledge-Graph-Driven Benchmarking and Collective Optimization for Physical Commonsense in Text-to-Image Generation [63.27323042387232]
  We introduce OmniPhys, a rigorous benchmark of 1,551 samples grounded in a Physical Knowledge Graph.<n>By aligning PhET simulations with standard curricula, OmniPhys operationalizes a knowledge-to-scenario pipeline that performs diagnostic stress tests.<n>We propose OmniPrompt, an iterative framework that treats physical feedback as a discrete optimization problem.
  arXiv  Detail & Related papers  (2026-07-28T12:27:25Z)
• Diagnosing Visual Ignorance in Vision-Language Models [29.2851901986069]
  Vision-Language Models (VLMs) frequently rely on language priors, producing confident answers that are weakly grounded in visual evidence.<n>In this work, we study language-prior reliance from both mechanistic and behavioral perspectives.<n>We find that language-prior reliance is a systematic routing failure affecting both model internals and benchmark validity.
  arXiv  Detail & Related papers  (2026-06-05T04:16:01Z)
• From Attenuation to Attention: Variational Information Flow Manipulation for Fine-Grained Visual Perception [12.4823697158657]
  Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding.<n>They frequently falter in fine-grained perception tasks that require identifying tiny objects or discerning subtle visual relationships.<n>We attribute this limitation to Visual Attenuation: a phenomenon where sparse fine-grained visual signals are prematurely suppressed or diluted by dominant textual tokens during network propagation.<n>Existing input-centric solutions fail to fundamentally reverse this intrinsic mechanism of information loss.<n>We propose the Variational Information Flow (VIF) framework to address this challenge.
  arXiv  Detail & Related papers  (2026-04-14T09:32:13Z)
• Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification [55.357038267439684]
  Visual Re-Examination (VRE) is a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs.<n>VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings.
  arXiv  Detail & Related papers  (2026-03-27T12:22:13Z)
• Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities? [61.533560295383786]
  Unified Multimodal Large Language Models (U-MLLMs) integrate understanding and generation within a single architecture.<n>We observe that U-MLLMs fail to maintain semantic equivalence when required to render the same results in the image modality.<n>We introduce VGUBench, a framework to decouple reasoning logic from generation fidelity.
  arXiv  Detail & Related papers  (2026-02-27T06:23:56Z)
• Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy [75.66913260900726]
  Reinforcement Learning with Verifiable Rewards has significantly advanced reasoning capabilities in Large Language Models.<n>Existing paradigms, driven by text-centric outcome rewards, encourage models to bypass visual perception.<n>We propose textbfThinking with Deltas, a framework driven by a textbfDifferential Visual Reasoning Policy.
  arXiv  Detail & Related papers  (2026-01-11T08:25:34Z)
• Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigation [27.62798534410331]
  We introduce Fine-grained Cross-modal Causal Tracing (FCCT) framework, which systematically quantifies the causal effects on visual object perception.<n>FCCT conducts fine-grained analysis covering the full range of visual and textual tokens, three core model components including multi-head self-attention (MHSA), feed-forward networks (FFNs), and hidden states, across all decoder layers.<n>Our analysis is the first to demonstrate that MHSAs of the last token in middle layers play a critical role in aggregating cross-modal information, while FFNs exhibit a three-stage hierarchical progression for the storage
  arXiv  Detail & Related papers  (2025-11-08T08:37:26Z)
• Robust Fairness Vision-Language Learning for Medical Image Analysis [11.848018894413556]
  We introduce a framework for ensuring robustness and fairness of Vision-Language Models.<n>This framework modifies the loss function at training by identifying and adjusting faulty image-text pairs.<n>We show up to a 8.6% improvement when looking at equity-scaled AUC.
  arXiv  Detail & Related papers  (2025-05-06T03:59:25Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment [57.0121616203175]
  We propose FiSAO, a novel self-alignment method that utilizes the model's own visual encoder as a fine-grained verifier to improve vision-language alignment.<n>By leveraging token-level feedback from the vision encoder, FiSAO significantly improves vision-language alignment, even surpassing traditional preference tuning methods that require additional data.
  arXiv  Detail & Related papers  (2024-10-18T03:34:32Z)
• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.