LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

• URL: http://arxiv.org/abs/2605.25979v1
• Date: Mon, 25 May 2026 15:54:04 GMT
• Title: LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
• Abstract summary: We present LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-OneVision model to date.<n>It achieves superior performance across a broad range of multimodal benchmarks.<n>A standout capability is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-language reasoning.
• Score: 107.63317552620231
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-language model in the LLaVA-OneVision series to date, achieving superior performance across a broad range of multimodal benchmarks. The model builds on a native OneVision-Encoder and incorporates Windowed Attention for efficient local computation while maintaining native resolution. Its key advance is codec-stream tokenization: it treats compressed video as a continuous bit-cost stream, where bit-cost dynamics determine adaptive temporal groups, and motion-residual cues select salient spatial evidence into compact visual canvases. This allocation concentrates a limited token budget on event-bearing content, enabling more stable long-video token compression than fixed groups of pictures. A shared 3D RoPE further places codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system. Furthermore, we build the LLaVA-OV-2 data and training stack around large-scale open supervision: approximately 8M re-captioned video samples for pretraining, a 4M-sample spatial corpus for fine-tuning. We also introduce JumpScore, a temporal-localization benchmark targeting fine-grained grounding in high-frequency, densely repeated motion, a regime underrepresented by existing video evaluations. A standout capability of LLaVA-OV-2 is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. On JumpScore, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B (30.1) by +44.8 points; under matched visual-token budgets on the same benchmark, codec-stream inputs improve temporal grounding over frame sampling by +9.7 points. Across standard benchmarks, LLaVA-OneVision-2-8B further outperforms Qwen3-VL-8B by +4.3 average points on video tasks, +5.3 on spatial tasks, and +15.6 average J&F on tracking tasks.

Related papers

• VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models [6.534399108411788]
  A class of Physical AI: Infrastructure AI relies on fixed cameras for open-loop logging insights.<n>We introduce VANTAGE-Bench, a benchmark measuring this "Infrastructure AI Gap"<n>We evaluate 17 models zero-shot, we find the shortfall relative to consumer-centric benchmarks is concentrated, not general.
  arXiv  Detail & Related papers  (2026-09-08T19:50:25Z)
• LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics [50.45297537821907]
  We introduce LeVJEPA, the first video encoder trained under LeJEPA's collapse-free objective.<n>A single encoder is trained with an invariance loss over global and local views of a clip.<n>LeVJEPA matches or surpasses V-JEPA 2 across ViT-S/B/L at 5.6 to 20.8x less pretraining compute.
  arXiv  Detail & Related papers  (2026-08-27T17:26:24Z)
• MOSS-VL Technical Report [66.58420246189313]
  We present MOSS-VL, an open vision-language model family that treats real-time interaction as a first-class capability.<n>It is co-designed across the stack: the language decoder attends to vision only through gated cross-attention.<n> Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-reasoning video sets.
  arXiv  Detail & Related papers  (2026-08-15T05:12:53Z)
• Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model [60.85752511708308]
  We present Mage-VL, an efficient streaming foundation model for real-time multimodal understanding and interaction.<n>At its core, our custom tokenizer, Mage-ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy-rich regions.<n>We establish AI4AI data pipelines encompassing prompt-code joint optimization for captioning and AI-driven performance diagnosis.
  arXiv  Detail & Related papers  (2026-07-27T17:59:53Z)
• UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling [33.6086006552446]
  We propose UniviewVLA, a unified multiview Vision-Language-Action model with world modeling.<n>By leveraging generated multiview future views, UniviewVLA reveals occluded cues and models future scene evolution.<n>UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free benchmarks.
  arXiv  Detail & Related papers  (2026-06-19T14:53:40Z)
• VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models [55.12929235609365]
  Visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision.<n>Existing implicit grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models.<n>We propose VEGA, a framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D.
  arXiv  Detail & Related papers  (2026-05-11T12:44:26Z)
• HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models [43.7747080569325]
  HieraVid is a hierarchical pruning framework that progressively and dynamically reduces visual redundancy.<n>We conduct extensive experiments on four widely used video understanding benchmarks to evaluate the effectiveness of HieraVid.<n>With only 30% of tokens retained, HieraVid achieves new state-of-the-art performance, while maintaining over 98% and 99% of the performance of LLaVA-Video-7B and LLaVA-OneVision-7B, respectively.
  arXiv  Detail & Related papers  (2026-04-02T10:40:15Z)
• See, Remember, Explore: A Benchmark and Baselines for Streaming Spatial Reasoning [41.278682336342165]
  We introduce S3-Bench, a benchmark suite for streaming spatial question answering with active exploration.<n>S3-Bench adopts a dual-domain design, combining a scalable simulator with controllable trajectories and exploration actions.<n>Our approach yields improvements of 8.8% and 13.3% on the simulated and real splits of S3-Eval, respectively.
  arXiv  Detail & Related papers  (2026-03-25T02:45:03Z)
• Open-o3 Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence [70.2803680525165]
  We introduce Open-o3 Video, a non-agent framework that integrates explicit evidence into video reasoning.<n>The model highlights key objects and bounding boxes alongside its answers, allowing reasoning to be grounded in concrete visual observations.<n>On V-STAR benchmark, Open-o3 Video achieves state-of-the-art performance, raising mAM by 14.4% and mL timestamp by 24.2%.
  arXiv  Detail & Related papers  (2025-10-23T14:05:56Z)
• EVA02-AT: Egocentric Video-Language Understanding with Spatial-Temporal Rotary Positional Embeddings and Symmetric Optimization [17.622013322533423]
  We introduce EVA02-AT, a suite of EVA02-based video-language foundation models tailored to egocentric video understanding tasks.<n> EVA02-AT efficiently transfers an image-based CLIP model into a unified video encoder via a single-stage pretraining.<n>We introduce the Symmetric Multi-Similarity (SMS) loss and a novel training framework that advances all soft labels for both positive and negative pairs.
  arXiv  Detail & Related papers  (2025-06-17T09:51:51Z)
• Improved Video VAE for Latent Video Diffusion Model [55.818110540710215]
  Video Autoencoder (VAE) aims to compress pixel data into low-dimensional latent space, playing an important role in OpenAI's Sora. Most of existing VAEs inflate a pretrained image VAE into the 3D causal structure for temporal-spatial compression. We propose a new KTC architecture and a group causal convolution (GCConv) module to further improve video VAE (IV-VAE)
  arXiv  Detail & Related papers  (2024-11-10T12:43:38Z)
• PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning [78.23573511641548]
  Vision-language pre-training has significantly elevated performance across a wide range of image-language applications. Yet, the pre-training process for video-related tasks demands exceptionally large computational and data resources. This paper investigates a straight-forward, highly efficient, and resource-light approach to adapting an existing image-language pre-trained model for video understanding.
  arXiv  Detail & Related papers  (2024-04-25T19:29:55Z)
• SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolution [18.332130780309797]
  This work develops a novel triple sparse convolution to reduce the computation costs of the overall video object segmentation framework. Experiments are conducted on two mainstream VOS datasets, including DAVIS and Youtube-VOS. Results show that, the proposed SpVOS achieves superior performance over other state-of-the-art sparse methods, and even maintains comparable performance.
  arXiv  Detail & Related papers  (2023-10-23T17:21:33Z)
• Scalable Neural Video Representations with Learnable Positional Features [73.51591757726493]
  We show how to train neural representations with learnable positional features (NVP) that effectively amortize a video as latent codes. We demonstrate the superiority of NVP on the popular UVG benchmark; compared with prior arts, NVP not only trains 2 times faster (less than 5 minutes) but also exceeds their encoding quality as 34.07rightarrow$34.57 (measured with the PSNR metric)
  arXiv  Detail & Related papers  (2022-10-13T08:15:08Z)
• AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition [44.10959567844497]
  This paper explores the unified formulation of spatial-temporal dynamic on top of the recently proposed AdaFocusV2 algorithm. AdaFocusV3 can be effectively trained by approximating the non-differentiable cropping operation with the computation of deep features.
  arXiv  Detail & Related papers  (2022-09-27T15:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.