Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modeling

• URL: http://arxiv.org/abs/2604.05445v1
• Date: Tue, 07 Apr 2026 05:28:39 GMT
• Title: Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modeling
• Abstract summary: VL-MDR (Vision-Language Multi-Dimensional Reward) is a framework that decomposes evaluation into granular, interpretable dimensions.<n>To support this, we curate a dataset of 321k vision-language preference pairs annotated across 21 fine-grained dimensions.
• Score: 14.249938992666202
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language reward modeling faces a dilemma: generative approaches are interpretable but slow, while discriminative ones are efficient but act as opaque "black boxes." To bridge this gap, we propose VL-MDR (Vision-Language Multi-Dimensional Reward), a framework that dynamically decomposes evaluation into granular, interpretable dimensions. Instead of outputting a monolithic scalar, VL-MDR employs a visual-aware gating mechanism to identify relevant dimensions and adaptively weight them (e.g., Hallucination, Reasoning) for each specific input. To support this, we curate a dataset of 321k vision-language preference pairs annotated across 21 fine-grained dimensions. Extensive experiments show that VL-MDR consistently outperforms existing open-source reward models on benchmarks like VL-RewardBench. Furthermore, we show that VL-MDR-constructed preference pairs effectively enable DPO alignment to mitigate visual hallucinations and improve reliability, providing a scalable solution for VLM alignment.

Related papers

• SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models [13.091215419258857]
  Vision-Language Models (VLMs) have demonstrated remarkable performance in processing and understanding both text and images.<n>We propose SlimVLM, a structured pruning framework designed to compress VLMs while preserving their task performance.
  arXiv  Detail & Related papers  (2026-08-04T12:34:46Z)
• Online Self-Calibration Against Hallucination in Vision-Language Models [23.13137973421435]
  Large Vision-Language Models (LVLMs) often suffer from hallucinations, generating descriptions that include visual details absent from the input image.<n>We propose textbfOnline textbfSelf-textbfCAlibtextbfRation (OSCAR), a framework that integrates Monte Carlo Tree Search with a Dual-Granularity Reward Mechanism to construct preference data.
  arXiv  Detail & Related papers  (2026-05-01T01:03:05Z)
• MURE: Hierarchical Multi-Resolution Encoding via Vision-Language Models for Visual Document Retrieval [111.99106496142474]
  Visual Document Retrieval (VDR) requires representations that capture both fine-grained visual details and global document structure.<n>Existing VDR models struggle to balance effectiveness and efficiency when processing high-resolution documents.<n>We propose a new X-VisEmb paradigm that progresses from multi-resolution sampling and encoding, through cross-granularity feature fusion, to adaptive representation distillation.
  arXiv  Detail & Related papers  (2026-03-07T15:17:22Z)
• VersaViT: Enhancing MLLM Vision Backbones via Task-Guided Optimization [87.26383908243878]
  We show that vision encoders within Multimodal Large Language Models exhibit deficiencies in their dense feature representations.<n>We propose VersaViT, a well-rounded vision transformer that instantiates a novel multi-task framework for collaborative post-training.
  arXiv  Detail & Related papers  (2026-02-10T16:08:19Z)
• FRISM: Fine-Grained Reasoning Injection via Subspace-Level Model Merging for Vision-Language Models [20.47311573790516]
  We propose FRISM (Fine-grained Reasoning Injection via Subspace-level model Merging), a fine-grained reasoning injection framework based on subspace-level model merging.<n>Experiments demonstrate that FRISM effectively improves reasoning capabilities without compromising the model's original visual capabilities.
  arXiv  Detail & Related papers  (2026-01-29T02:36:19Z)
• Youtu-VL: Unleashing Visual Potential via Unified Vision-Language Supervision [79.06371915084833]
  We introduce Youtu-VL, a framework leveraging the Vision-Language Unified Autoregressive Supervision (VLUAS) paradigm.<n>Youtu-VL applies unified autoregressive supervision to both visual details and linguistic content.<n>We extend this paradigm to encompass vision-centric tasks, enabling a standard VLM to perform vision-centric tasks without task-specific additions.
  arXiv  Detail & Related papers  (2026-01-27T17:01:16Z)
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling [84.51372201195132]
  CronusVLA is a unified framework that extends single-frame VLA models to the multi-frame paradigm.<n>CronusVLA achieves leading performance and superior robustness, with a 70.9% success rate.<n>These results highlight the potential of efficient multi-frame adaptation in VLA models for more powerful and robust real-world deployment.
  arXiv  Detail & Related papers  (2025-06-24T17:30:27Z)
• Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning [23.851747078717473]
  We introduce textbfValue-guided Inference with Margin-based Reward (ViMaR), a two-stage inference framework that improves both efficiency and output fidelity.<n>ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$times$ speedup.
  arXiv  Detail & Related papers  (2025-06-18T17:23:36Z)
• Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization [40.77611907215627]
  Large Vision Language Models (VLMs) are prone to significant hallucinations, particularly in the form of cross-modal inconsistencies.<n>We introduce Re-Align, a novel alignment framework that leverages image retrieval to construct a dual-preference dataset.<n>We also introduce rDPO, an extension of the standard direct preference optimization that incorporates an additional visual preference objective during fine-tuning.
  arXiv  Detail & Related papers  (2025-02-18T18:59:57Z)
• LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models [57.92316645992816]
  Spatial reasoning is a fundamental aspect of human cognition, enabling intuitive understanding and manipulation of objects in three-dimensional space.<n>We introduce LayoutVLM, a framework and scene layout representation that exploits the semantic knowledge of Vision-Language Models (VLMs)<n>We demonstrate that fine-tuning VLMs with the proposed scene layout representation extracted from existing scene datasets can improve their reasoning performance.
  arXiv  Detail & Related papers  (2024-12-03T06:15:04Z)
• Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance [51.30560006045442]
  Image-gRounded guIdaNcE (MARINE) is a framework that is both training-free and API-free.<n>MARINE effectively and efficiently reduces object hallucinations during inference by introducing image-grounded guidance to LVLMs.<n>Our framework's flexibility further allows for the integration of multiple vision models, enabling more reliable and robust object-level guidance.
  arXiv  Detail & Related papers  (2024-02-13T18:59:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.