Daily Paper Briefing

Abstract Overview

InternVideo3 is a framework for long-horizon video understanding that reformulates multimodal understanding as Multimodal Contextual Reasoning (MCR), a closed-loop process over an evolving shared context. In this formulation, observations, intermediate reasoning, tool actions, feedback, and memory are all appended to a unified context so the model can accumulate evidence, revise beliefs, and verify answers over time. To make such long rollouts practical, the paper introduces Multimodal Multi-head Latent Attention (M2LA), which compresses KV-cache states while preserving the full multimodal token stream. The system is trained with continued pretraining after attention conversion, short-to-long supervised fine-tuning, rule-based reinforcement learning, and on-policy distillation. Experiments show strong open-weight performance on long-video, short-video, and spatiotemporal benchmarks, with especially strong results on several long-horizon tasks.

Novelty

The paper’s main distinction is the combination of a closed-loop multimodal reasoning formulation for video, MCR, with an attention reparameterization, M2LA, that targets long-context multimodal rollouts by compressing cached attention states without dropping tokens. It also pairs this with a staged long-horizon training recipe and a video-agent instantiation that integrates retrieval, grounding, and verification tools within the same evolving context.

Results

InternVideo3 reports the best open-weight results in the paper’s tables on multiple long-video benchmarks, including Video-MME, MLVU, VRBench, and EgoSchema, and it also leads the listed open-weight models in Short QA average on short-video evaluation. On spatiotemporal evaluation, it achieves the best listed open-weight results on all three temporal grounding benchmarks and strong spatial results, especially on VSIBench and MMSIBench-Video. M2LA also improves long-context inference efficiency, with reported decode throughput gains that increase with context length and execution remaining feasible at 768K prefill tokens, where the original model runs out of memory earlier.

Key Points

1. InternVideo3 models video understanding as a recursive evidence-gathering process by placing observations, reasoning traces, tool outputs, feedback, and memory into a shared evolving context.
2. The proposed M2LA attention mechanism reduces KV-cache cost for long multimodal rollouts while preserving the full token stream, supporting substantially longer contexts and faster long-context decoding.
3. Empirically, the model shows strong open-weight benchmark performance, especially on long-horizon video tasks and temporal grounding, and the paper also demonstrates a video-agent setup with retrieval and verification tools.

References

• arXiv: https://arxiv.org/abs/2606.12195v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2606.12195v1
• Hugging Face Papers: https://huggingface.co/papers/2606.12195