ReasonX: MLLM-Guided Intrinsic Image Decomposition

• URL: http://arxiv.org/abs/2512.04222v1
• Date: Wed, 03 Dec 2025 19:44:07 GMT
• Title: ReasonX: MLLM-Guided Intrinsic Image Decomposition
• Abstract summary: Intrinsic image decomposition aims to separate images into physical components such as albedo, depth, normals, and illumination.<n>We propose ReasonX, a novel framework that leverages a large language model (MLLM) as a perceptual judge providing relative intrinsic comparisons.<n>Our framework aligns conditional intrinsic predictors by rewarding agreement between the judge's assessments and analytically derived relations from the model's outputs.
• Score: 41.961378846441015
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intrinsic image decomposition aims to separate images into physical components such as albedo, depth, normals, and illumination. While recent diffusion- and transformer-based models benefit from paired supervision from synthetic datasets, their generalization to diverse, real-world scenarios remains challenging. We propose ReasonX, a novel framework that leverages a multimodal large language model (MLLM) as a perceptual judge providing relative intrinsic comparisons, and uses these comparisons as GRPO rewards for fine-tuning intrinsic decomposition models on unlabeled, in-the-wild images. Unlike RL methods for generative models, our framework aligns conditional intrinsic predictors by rewarding agreement between the judge's relational assessments and analytically derived relations from the model's outputs. ReasonX is model-agnostic and can be applied to different intrinsic predictors. Across multiple base architectures and modalities, ReasonX yields significant improvements, including 9-25% WHDR reduction on IIW albedo and up to 46% depth accuracy gains on ETH3D, highlighting the promise of MLLM-guided comparative supervision to bridge low- and high-level vision reasoning.

Related papers

• Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment [48.882448108261826]
  Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception.<n>Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores.<n>We introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score.
  arXiv  Detail & Related papers  (2026-05-07T21:38:09Z)
• ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding [23.27523995613204]
  We propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations.<n>By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns.<n>Our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.
  arXiv  Detail & Related papers  (2026-04-20T04:55:11Z)
• CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling [61.75914342638658]
  CDRRM (Contrast-Driven Reward Model) is a framework for high-quality rubric generation and guided preference judgment.<n>This work offers a scalable, interpretable, and data-efficient path for reward modeling.
  arXiv  Detail & Related papers  (2026-03-09T07:15:23Z)
• Unified Personalized Reward Model for Vision Generation [27.496220369122494]
  We propose UnifiedReward-Flex, a unified personalized reward model for vision generation.<n>We first distill structured, high-quality reasoning traces from advanced closed-source VLMs to bootstrap SFT.<n>We then perform direct preference optimization (DPO) on carefully curated preference pairs to further strengthen reasoning fidelity and discriminative alignment.
  arXiv  Detail & Related papers  (2026-02-02T17:44:21Z)
• PDE-Constrained Optimization for Neural Image Segmentation with Physics Priors [0.0]
  Images of microscopy images constitutes an ill-posed inverse problem due to measurement noise, weak object boundaries, and limited labeled data.<n>In this work, image segmentation is formulated as a PDE-constrained optimization problem that integrates physically motivated priors into deep learning models.<n>Experiments are conducted on the LIVECell dataset, a high-quality, manually annotated collection of phase-contrast microscopy images.
  arXiv  Detail & Related papers  (2026-02-01T07:28:14Z)
• Mixture-of-Experts Models in Vision: Routing, Optimization, and Generalization [0.0]
  We study MoE behavior in an image classification setting, focusing on predictive performance, expert utilization, and generalization.<n>We compare dense, SoftMoE, and SparseMoE classifier heads on the CIFAR10 dataset under comparable model capacity.<n>Both MoE variants achieve slightly higher validation accuracy than the dense baseline while maintaining balanced expert utilization through regularization.<n>We find that SoftMoE exhibits higher sharpness by these metrics, while Dense and SparseMoE lie in a similar curvature regime, despite all models achieving comparable generalization performance.
  arXiv  Detail & Related papers  (2026-01-21T14:22:25Z)
• Latent Dirichlet Transformer VAE for Hyperspectral Unmixing with Bundled Endmembers [1.9336815376402718]
  We propose the Latent Dirichlet Transformer Variational Autoencoder (LDVAE-T) for hyperspectral unmixing.<n>Our model combines the global context modeling capabilities of transformer architectures with physically meaningful constraints imposed by a Dirichlet prior in the latent space.<n>We evaluate our approach on three benchmark datasets, Samson, Jasper Ridge, and HYDICE Urban.
  arXiv  Detail & Related papers  (2025-11-21T20:15:37Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization [63.169050703903515]
  We propose Aes-R1, a comprehensive aesthetic reasoning framework with reinforcement learning (RL)<n>Aes-R1 integrates a pipeline, AesCoT, to construct and filter high-quality chain-of-thought aesthetic reasoning data.<n>Experiments demonstrate that Aes-R1 improves the backbone's average PLCC/SRCC by 47.9%/34.8%.
  arXiv  Detail & Related papers  (2025-09-26T04:55:00Z)
• FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities [76.46448367752944]
  multimodal large language models (MLLMs) unify visual understanding and image generation within a single framework.<n>Most existing MLLMs rely on autore (AR) architectures, which impose inherent limitations on future development.<n>We introduce FUDOKI, a unified multimodal model purely based on discrete flow matching.
  arXiv  Detail & Related papers  (2025-05-26T15:46:53Z)
• Hybrid AI-Physical Modeling for Penetration Bias Correction in X-band InSAR DEMs: A Greenland Case Study [1.4711955189581167]
  We propose an integrated correction framework that combines parametric physical modeling with machine learning.<n>Experiments on Greenland's ice sheet using TanDEM-X data show that the proposed hybrid model corrections significantly reduce the mean and standard deviation of DEM errors.
  arXiv  Detail & Related papers  (2025-04-11T18:24:22Z)
• Towards Fair and Robust Face Parsing for Generative AI: A Multi-Objective Approach [10.00430939898858]
  We propose a multi-objective learning framework that optimize accuracy, fairness, and robustness in face parsing.<n>Our results show that fairness-aware and robust segmentation improves photorealism and consistency in face generation.<n>Our findings demonstrate that multi-objective face parsing improves demographic consistency and robustness, leading to higher-quality GAN-based synthesis.
  arXiv  Detail & Related papers  (2025-02-06T00:41:35Z)
• Modality-Fair Preference Optimization for Trustworthy MLLM Alignment [22.093944381988496]
  Multimodal large language models (MLLMs) have achieved remarkable success across various tasks.<n>However, separate training of visual and textual encoders often results in a misalignment of the modality.<n>These inaccuracies severely undermine the trustworthiness of MLLMs in real-world applications.
  arXiv  Detail & Related papers  (2024-10-20T08:56:52Z)
• Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation [15.411325887412413]
  This paper proposes a novel model named "Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment" (FSA-CDM) FSA-CDM introduces contrastive positive/negative samples into the diffusion model to boost performance for markup-to-image generation. Experiments are conducted on four benchmark datasets from different domains.
  arXiv  Detail & Related papers  (2023-08-02T13:43:03Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• A Model for Multi-View Residual Covariances based on Perspective Deformation [88.21738020902411]
  We derive a model for the covariance of the visual residuals in multi-view SfM, odometry and SLAM setups. We validate our model with synthetic and real data and integrate it into photometric and feature-based Bundle Adjustment.
  arXiv  Detail & Related papers  (2022-02-01T21:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.