DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

• URL: http://arxiv.org/abs/2607.08434v1
• Date: Thu, 09 Jul 2026 12:54:05 GMT
• Title: DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models
• Abstract summary: Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states.<n>This full-image generation paradigm introduces substantial visual-token redundancy and dilutes supervision on sparse yet reasoning-critical state transitions.<n>We propose DeltaV, a ULMM that replaces full-image generation with visual updates.
• Score: 61.3037243251433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visual state as a full image. This full-image generation paradigm introduces substantial visual-token redundancy and dilutes supervision on sparse yet reasoning-critical state transitions. We propose DeltaV, a ULMM that replaces full-image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update tokens that capture the visual changes across reasoning steps, avoiding repeated modeling of unchanged content. To align the token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable reasoning, we further construct StructCoT, a large-scale interleaved multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual-update paradigm reduces newly generated visual tokens by 55.6\% on average without compromising reconstruction fidelity, and improves multimodal reasoning by 3.3\% over full-image generation. Trained with StructCoT and large-scale multimodal data, DeltaV-2B further outperforms substantially larger open-source models by 8.4\% on in-domain multimodal reasoning evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9\% on external multimodal reasoning and understanding benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie-W/DeltaV.

Related papers

• Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling [22.020464703520148]
  We introduce DenseAR, a new generative paradigm that reformulates autoregressive image generation as coarse-to-fine next-dense-stride prediction.<n>Our key insight is that traversing a single-scale latent grid with progressively denser strides naturally captures the transition from global structure to fine detail.
  arXiv  Detail & Related papers  (2026-07-10T18:32:27Z)
• Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification [80.62512020268626]
  UniAR is a unified autoregressive framework where a single visual tokenizer serves as the key bridge between understanding and generation.<n>UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme.<n>A diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images.
  arXiv  Detail & Related papers  (2026-06-16T17:59:22Z)
• LoMo: Local Modality Substitution for Deeper Vision-Language Fusion [10.837487434757882]
  Local Modality Substitution (LoMo) is a lightweight, architecture-agnostic data curation paradigm designed to provide supervision for cross-modal representational invariance between semantically equivalent text and image carriers.<n>LoMo significantly improves overall multimodal reasoning and yields deeper cross-modal fusion. Specifically, it delivers consistent gains across foundational models, improving over standard SFT by 2.67 points on LLaVA-Vision-1.5-8B and 2.82 points on Qwen3.5-9B.
  arXiv  Detail & Related papers  (2026-05-28T17:27:55Z)
• Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models [58.9387276447485]
  Vision Inference Former (VIF) is a lightweight architectural module that establishes a bridge between pure visual representations and the model's output space.<n>We conduct experiments on 14 benchmark tasks covering general reasoning, OCR, table understanding, vision-centric evaluation, and hallucination.
  arXiv  Detail & Related papers  (2026-05-18T10:04:22Z)
• Revisiting Change VQA in Remote Sensing with Structured and Native Multimodal Qwen Models [2.4564286418294468]
  Change visual question answering (Change VQA) addresses the problem of answering natural-language questions about semantic changes between bi-temporal remote sensing (RS) images.<n>Recent vision-language models (VLMs) have been studied for temporal RS image understanding, but Change VQA remains underexplored in the context of modern multimodal models.<n>We compare Qwen3-VL, which follows a structured vision-language pipeline with multi-depth visual conditioning and a full-attention decoder, with Qwen3.5, a native multimodal model that combines a single-stage alignment with a hybrid decoder backbone.
  arXiv  Detail & Related papers  (2026-04-20T15:47:52Z)
• Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation [66.53544128707817]
  Cheers is a unified multimodal model that decouples patch-level details from semantic representations.<n> Cheers matches or surpasses advanced UMMs in both visual understanding and generation.
  arXiv  Detail & Related papers  (2026-03-13T08:55:27Z)
• Kelix Technical Report [86.64551727600104]
  We present Kelix, a fully discrete autoregressive unified model that closes the understanding gap between discrete and continuous visual representations.<n>Recent work has explored discrete visual tokenization to enable fully autoregressive multimodal modeling.
  arXiv  Detail & Related papers  (2026-02-10T14:48:26Z)
• Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting [53.332533610841885]
  Time series can be transformed into images and texts, offering multi-modal views (MMVs) of the same underlying signal.<n>These MMVs can reveal complementary patterns and enable the use of powerful pre-trained large models, such as large vision models (LVMs) for long-term time series forecasting (LTSF)<n>We propose DMMV, a novel decomposition-based multi-modal view framework that leverages trend-seasonal decomposition and a novel backcast-residual based adaptive decomposition to integrate MMVs for LTSF.
  arXiv  Detail & Related papers  (2025-05-29T20:55:24Z)
• v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning [27.688428439248607]
  We introduce v1, a lightweight extension that enables active visual referencing through a simple point-and-copy approach.<n>This allows the model to identify relevant image patches and copy their embeddings back into the reasoning stream.<n>Our pointing strategy lets the MLLM directly select image patches using their semantic representations as keys, keeping perceptual evidence embedded in the same space as the model's reasoning.
  arXiv  Detail & Related papers  (2025-05-24T19:30:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.