ParVL: Parallel Scaling and Expandable Compute Allocation for Multimodal LLMs

• URL: http://arxiv.org/abs/2608.04010v1
• Date: Tue, 04 Aug 2026 17:59:58 GMT
• Title: ParVL: Parallel Scaling and Expandable Compute Allocation for Multimodal LLMs
• Abstract summary: We introduce the Parallel Vision-Language (ParVL) scaling framework for Multimodal Large Language Models.<n>ParVL scales parallel computation by reusing the existing ViT and LLM backbone parameters across multiple vision and language branches.<n>We show that ParVL improves overall multimodal performance over same-recipe single-branch baselines.
• Score: 48.87461975674182
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing scaling strategies for Multimodal Large Language Models (MLLMs) typically expand either model parameters or sequential inference computation, incurring substantial memory or latency overhead. More importantly, most existing methods fail to alter the rigid, fixed computation allocation between the Vision Transformer and the Large Language Model components, limiting task-specific optimization. To address this, we introduce the Parallel Vision-Language (ParVL) scaling framework for MLLMs, which scales parallel computation by reusing the existing ViT and LLM backbone parameters across multiple vision and language branches. This framework raises a central question: given a fixed backbone parameter budget, how should additional shared-backbone computation be allocated between the vision and language modalities? We instantiate each parallel computational stream with branch-specific prefix parameters over a shared backbone, and train the entire model end-to-end via full-parameter supervised fine-tuning on roughly 13B tokens. We systematically study the computation-allocation trade-off between the ViT encoder and LLM decoder. ParVL improves overall multimodal performance over same-recipe single-branch baselines, and the best evaluated vision--language allocation varies across tasks. Code is available at https://github.com/YangYangGirl/ParVL.

Related papers

• Look Less, Think Faster: Joint Token-Compute Adaptation for Multimodal LLMs [16.64366862436724]
  Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance across vision-language tasks.<n>Their high inference cost, arising from both the large number of input visual tokens and the heavy computation of the large language model (LLM), remains a key barrier to practical deployment.<n>We propose SmartVL, a unified adaptive inference framework that jointly controls vision token number and model compute capability.
  arXiv  Detail & Related papers  (2026-07-22T16:43:27Z)
• An MLIR-Based Compilation Method for Large Language Models [9.14762452723213]
  This paper presents an MLIR based compilation method for large language models.<n>A model is first represented as TopOp, then lowered layer by layer to TpuOp, and finally a deployable binary is generated.<n>The method has been implemented in the TPU-MLIR compiler and the LLM-TPU deployment project.
  arXiv  Detail & Related papers  (2026-07-17T11:24:45Z)
• Towards Large Model Feature Coding [90.89390892560094]
  We present a benchmark and evaluation framework for large model feature coding (LaMoFC)<n>We first build the feature dataset LaMoFCBench, covering diverse task requirements across 4 categories and 16 scenarios.<n>We then specify representative split points according to practical application scenarios to extract intermediate features.
  arXiv  Detail & Related papers  (2026-05-20T13:39:48Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion [91.35078719566472]
  Vision-Language Models (VLMs) create a severe visual feature bottleneck by using a crude, asymmetric connection.<n>We introduce Cross-Layer Injection (CLI), a novel and lightweight framework that forges a dynamic many-to-many bridge between the two modalities.
  arXiv  Detail & Related papers  (2026-01-15T18:59:10Z)
• Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models [59.21082876068122]
  Large language models (LLMs) have demonstrated impressive translation capabilities even without being explicitly trained on parallel data.<n>Recent work suggests that it is actually caused by incidental bilingual signals present in the training data.<n>Various methods have been proposed to maximize the utility of parallel data to enhance the multilingual capabilities of multilingual encoder-based and encoder-decoder language models.
  arXiv  Detail & Related papers  (2025-06-16T02:21:15Z)
• ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning [38.26304604660713]
  ADEM-VL is an efficient vision-language method that tunes models based on pretrained large language models. Our framework surpasses existing methods by an average accuracy of 0.77% on ScienceQA dataset.
  arXiv  Detail & Related papers  (2024-10-23T11:31:06Z)
• LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism [12.521026493432181]
  Existing large language models (LLMs) cannot efficiently serve variable-length requests in different phases. We propose a new parallelism paradigm, elastic sequence parallelism (ESP), to adapt to the variance between different requests and phases. LoongServe improves the maximum throughput by up to 3.85$times$ compared to the chunked prefill and 5.81$times$ compared to the prefill-decoding disaggregation.
  arXiv  Detail & Related papers  (2024-04-15T07:45:04Z)
• HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models [70.25499865569353]
  We introduce HyperLLaVA, which involves adaptive tuning of the projector and LLM parameters, in conjunction with a dynamic visual expert and language expert. Our solution significantly surpasses LLaVA on existing MLLM benchmarks, including MME, MMBench, SEED-Bench, and LLaVA-Bench.
  arXiv  Detail & Related papers  (2024-03-20T09:42:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.