Reliability-Prioritized Fine-Grained Generation in Multimodal Large

• URL: http://arxiv.org/abs/2606.29573v2
• Date: Wed, 01 Jul 2026 16:14:16 GMT
• Title: Reliability-Prioritized Fine-Grained Generation in Multimodal Large
• Abstract summary: We show that generating fine-grained descriptions is more error-prone than coarse-grained generation.<n>We propose a hierarchy-aware evaluation algorithm, which assesses both whether model predictions are visually correct and how specific the correct predictions are.<n>Experiments on textscGranFact show that our method improves fine-grained generation while preserving reliability.
• Score: 50.26126844186488
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multimodal large language models (MLLMs) are increasingly expected to generate fine-grained descriptions of visual content. However, we observe and theoretically show that generating fine-grained responses poses a reliability challenge, \textit{i.e.}, fine-grained generation is more error-prone than coarse-grained generation. This phenomenon suggests that models should generate the finest description that remains reliable rather than simply produce more specific outputs. To investigate this problem, we develop \textsc{GranFact}, a granularity-aware benchmark consisting of expert-verified multi-object images with coarse-to-fine category annotations. Then, we design a hierarchy-aware evaluation algorithm, which assesses both whether model predictions are visually correct and how specific the correct predictions are. We also propose a reliability-prioritized preference optimization method based on Direct Preference Optimization, which penalizes unreliable fine-grained claims while rewarding reliable specificity. Experiments on \textsc{GranFact} show that our method improves fine-grained generation while preserving reliability. Code and data are available \href{https://github.com/WeiWu2025/GranFact}{here}.

Related papers

• HoloRec: Holistic Encoding and Interleaved Reasoning for Generative Recommendation [57.802158017741824]
  HoloRec is an endogenous chain-of-thought recommendation mechanism that unifies representation, reasoning, and generation.<n> Experiments on multiple public recommendation datasets demonstrate that HoloRec consistently outperforms baselines.
  arXiv  Detail & Related papers  (2026-06-13T14:52:03Z)
• Verifiable Reasoning for LLM-based Generative Recommendation [106.7765000777685]
  Reasoning in Large Language Models (LLMs) has recently shown strong potential in enhancing generative recommendation.<n>We propose a novel textbftextitreason-verify-recommend paradigm, which interleaves reasoning with verification to provide reliable feedback.<n>Experiments on four real-world datasets demonstrate that VRec substantially enhances recommendation effectiveness and scalability without compromising efficiency.
  arXiv  Detail & Related papers  (2026-03-08T16:55:45Z)
• Specificity-aware reinforcement learning for fine-grained open-world classification [54.85385270439992]
  Classifying fine-grained visual concepts under open-world settings demands models to be both accurate and specific.<n>We propose a novel specificity-aware reinforcement learning framework, SpeciaRL, to fine-tune reasoning LMMs on fine-grained image classification.
  arXiv  Detail & Related papers  (2026-03-03T17:52:39Z)
• SimGR: Escaping the Pitfalls of Generative Decoding in LLM-based Recommendation [68.00727783181289]
  A core objective in recommender systems is to accurately model the distribution of user preferences over items to enable personalized recommendations.<n>We observe that existing methods inevitably introduce systematic bias when estimating item-level preference distributions.<n>We propose textbfSimply textbfGenerative textbfRecommendation (textbfSimGR), a framework that directly models item-level preference distributions in a shared latent space.
  arXiv  Detail & Related papers  (2026-02-08T07:26:52Z)
• GenProve: Learning to Generate Text with Fine-Grained Provenance [14.66501508318216]
  We introduce Generation-time Fine-grained Provenance, a task where models must generate fluent answers while simultaneously producing structured, sentence-level provenance triples.<n>By optimizing a composite reward for answer fidelity and provenance correctness, GenProve significantly outperforms 14 strong LLMs in joint evaluation.
  arXiv  Detail & Related papers  (2026-01-08T13:30:30Z)
• PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations [52.67948063133533]
  Generative Recommendation has emerged as a promising paradigm, reformulating recommendation as a sequence-to-sequence generation task over hierarchical Semantic IDs.<n>Existing methods suffer from a critical issue we term Semantic Drift, where errors in early, high-level tokens irreversibly divert the generation trajectory into irrelevant semantic subspaces.<n>We propose Promise, a novel framework that integrates dense, step-by-step verification into generative models.
  arXiv  Detail & Related papers  (2026-01-08T07:38:46Z)
• OSPO: Object-centric Self-improving Preference Optimization for Text-to-Image Generation [9.584960092259033]
  Object-centric Self-improving Preference Optimization (OSPO) is a self-improving framework for enhancing object-level text-image alignment.<n>OSPO significantly improves fine-grained alignment in text-to-image generation.
  arXiv  Detail & Related papers  (2025-05-28T03:45:42Z)
• Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction [0.8594140167290097]
  We develop conformal prediction methods for constructing valid confidence sets in multiclass and multilabel problems. By leveraging ideas from quantile regression, we build methods that always guarantee correct coverage but additionally provide conditional coverage for both multiclass and multilabel prediction problems.
  arXiv  Detail & Related papers  (2020-04-21T17:45:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.