Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

• URL: http://arxiv.org/abs/2605.25820v1
• Date: Mon, 25 May 2026 13:16:51 GMT
• Title: Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models
• Abstract summary: Diffusion-based multimodal large language models (dMLLMs) decode by iteratively predicting tokens at multiple masked positions in parallel.<n>Existing confidence-based decoding ranks masked positions independently and commits the top-K positions, largely ignoring whether the committed tokens provide complementary visual grounding.<n>We propose Visual-Redundancy-Controlled Decoding (VRCD), a training-free inference-time decoding method that uses token-to-image attention to prioritize visually complementary positions.
• Score: 9.95016675032699
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iteratively predicting tokens at multiple masked positions in parallel. This turns each decoding step into a position-selection problem: the model must choose not only which predictions are reliable in isolation, but also which positions should be committed together as context for later decoding steps. Existing confidence-based decoding ranks masked positions independently and commits the top-K positions, largely ignoring whether the committed tokens provide complementary visual grounding. We identify a step-level limitation of this strategy in multimodal settings: high-confidence tokens selected in the same step can rely on overlapping visual grounding, introducing visual redundancy among the committed tokens and leaving less complementary visual grounding available for later decoding. To quantify this effect, we introduce the Visual Redundancy Index (VRI), which measures visual grounding overlap among tokens committed in parallel. To control this redundancy during decoding, we propose Visual-Redundancy-Controlled Decoding (VRCD), a training-free inference-time decoding method that uses token-to-image attention to prioritize visually complementary positions. Across diverse multimodal benchmarks, VRCD reduces visual redundancy and remaining-position entropy with modest runtime overhead. In longer decoding experiments, it also achieves relative accuracy gains of up to 18.8% on M^3CoT and 6.9% on MMBench over confidence-based decoding. Code will be released at https://github.com/infiniteYuanyl/VRCD.

Related papers

• Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models [20.55764219638966]
  Diffusion multimodal large language models (dMLLMs) have emerged as a new decoding paradigm for multimodal generation.<n>We propose VIG-Sampler, which prioritizes tokens based on their attention to image tokens.<n>Experiments on 7 captioning and VQA benchmarks with 3 open-source dMLLMs demonstrate the effectiveness of VIG-Sampler.
  arXiv  Detail & Related papers  (2026-08-27T03:46:27Z)
• Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification [80.62512020268626]
  UniAR is a unified autoregressive framework where a single visual tokenizer serves as the key bridge between understanding and generation.<n>UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme.<n>A diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images.
  arXiv  Detail & Related papers  (2026-06-16T17:59:22Z)
• Diffusion Large Language Models for Visual Speech Recognition [56.81307584718608]
  We propose the first Diffusion Large Language Model (DLLM)-based Visual Speech Recognition (VSR) framework.<n>DLLM-VSR commits high-confidence positions early and uses the committed tokens as bidirectional context to refine ambiguous ones.<n>We develop length-guided candidate decoding, which uses video duration to construct plausible transcript-length hypotheses.<n>The proposed method achieves a state-of-the-art WER of 19.5% on LRS3 using only its labeled training data.
  arXiv  Detail & Related papers  (2026-05-27T13:22:08Z)
• LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding [80.49137401046201]
  We introduce a unified generative grounding and detection framework based on Parallel Box Decoding (PBD)<n>We show that PBD improves both decoding throughput and localization accuracy.<n>We further develop a scalable data engine and curate LocateAnything-Data, a large-scale dataset with more than 138 million training samples.
  arXiv  Detail & Related papers  (2026-05-26T17:59:12Z)
• A More Word-like Image Tokenization for MLLMs [26.120899392740203]
  We propose a novel Disentangled Visual Tokenization (DiVT) that clusters patch embeddings into coherent semantic units.<n>Across diverse multimodal benchmarks, DiVT matches or surpasses baselines with significantly fewer visual tokens.
  arXiv  Detail & Related papers  (2026-05-18T07:09:46Z)
• PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding [32.667256256847246]
  Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences.<n>We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes.
  arXiv  Detail & Related papers  (2026-05-15T04:43:02Z)
• CoME-VL: Scaling Complementary Multi-Encoder Vision-Language Learning [79.07656918674684]
  CoME-VL is a modular fusion framework that integrates a contrastively trained vision encoder with a self-supervised DINO encoder.<n>Our method achieves state-of-the-art performance on RefCOCO for detection while improving over the baseline by a large margin.
  arXiv  Detail & Related papers  (2026-04-03T17:59:51Z)
• Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation [66.53544128707817]
  Cheers is a unified multimodal model that decouples patch-level details from semantic representations.<n> Cheers matches or surpasses advanced UMMs in both visual understanding and generation.
  arXiv  Detail & Related papers  (2026-03-13T08:55:27Z)
• VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction [83.50898344094153]
  VQRAE produces Continuous semantic features for image understanding and tokens for visual generation within a unified tokenizer.<n>Design enables negligible semantic information for maintaining the ability of multimodal understanding, discrete tokens.<n>VQRAE presents competitive performance on several benchmarks of visual understanding, generation and reconstruction.
  arXiv  Detail & Related papers  (2025-11-28T17:26:34Z)
• ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution [71.69364653858447]
  Existing Multimodal Large Language Models (MLLMs) suffer from increased inference costs due to the additional vision tokens introduced by image inputs.<n>We propose Visual Consistency Learning (ViCO), a novel training algorithm that enables the model to represent images of varying complexities using different numbers of vision tokens.<n> Experimental results demonstrate that our method can reduce the number of vision tokens by up to 50% while maintaining the model's perception, reasoning, and OCR capabilities.
  arXiv  Detail & Related papers  (2025-10-14T17:58:10Z)
• METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models [92.37117312251755]
  We propose a progressive pruning framework, namely Multi-Encoder collaboraTivE tOken pRuning (METEOR)<n>For multi-vision encoding, we discard redundant tokens within each encoder via a rank guided collaborative token assignment strategy.<n>For multi-vision fusion, we combine the visual features from different encoders while reducing cross-encoder redundancy with cooperative pruning.
  arXiv  Detail & Related papers  (2025-07-28T13:50:53Z)
• ToDRE: Visual Token Pruning via Diversity and Task Awareness for Efficient Large Vision-Language Models [59.47738955960352]
  ToDRE is a two-stage and training-free token compression framework.<n>It achieves superior performance by pruning tokens based on token Diversity and token-task RElevance.
  arXiv  Detail & Related papers  (2025-05-24T15:47:49Z)
• Consistent Multiple Sequence Decoding [36.46573114422263]
  We introduce a consistent multiple sequence decoding architecture. This architecture allows for consistent and simultaneous decoding of an arbitrary number of sequences. We show the efficacy of our consistent multiple sequence decoder on the task of dense relational image captioning.
  arXiv  Detail & Related papers  (2020-04-02T00:43:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.