V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning

• URL: http://arxiv.org/abs/2603.14482v2
• Date: Tue, 17 Mar 2026 14:12:31 GMT
• Title: V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning
• Abstract summary: V-JEPA 2.1 is a family of self-supervised models that learn dense, high-quality visual representations for both images and videos.<n>We show that V-JEPA 2.1 significantly advances the state-of-the-art in dense visual understanding and world modeling.
• Score: 46.89986408533846
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present V-JEPA 2.1, a family of self-supervised models that learn dense, high-quality visual representations for both images and videos while retaining strong global scene understanding. The approach combines four key components. First, a dense predictive loss uses a masking-based objective in which both visible and masked tokens contribute to the training signal, encouraging explicit spatial and temporal grounding. Second, deep self-supervision applies the self-supervised objective hierarchically across multiple intermediate encoder layers to improve representation quality. Third, multi-modal tokenizers enable unified training across images and videos. Finally, the model benefits from effective scaling in both model capacity and training data. Together, these design choices produce representations that are spatially structured, semantically coherent, and temporally consistent. Empirically, V-JEPA 2.1 achieves state-of-the-art performance on several challenging benchmarks, including 7.71 mAP on Ego4D for short-term object-interaction anticipation and 40.8 Recall@5 on EPIC-KITCHENS for high-level action anticipation, as well as a 20-point improvement in real-robot grasping success rate over V-JEPA-2 AC. The model also demonstrates strong performance in robotic navigation (5.687 ATE on TartanDrive), depth estimation (0.307 RMSE on NYUv2 with a linear probe), and global recognition (77.7 on Something-Something-V2). These results show that V-JEPA 2.1 significantly advances the state of the art in dense visual understanding and world modeling.

Related papers

• PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models [60.567497085815724]
  PhysBrain 1.5 is a unified model for understanding physical environments, generating actions, and predicting future states.<n>Pre-training draws its embodied supervision entirely from human interaction videos.<n>It achieves the best open-source results on 14 benchmarks while retaining general multimodal capabilities.
  arXiv  Detail & Related papers  (2026-09-14T03:25:46Z)
• Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision [26.793499191581315]
  Cross-embodiment transfer in vision-language-action (VLA) models remains challenging.<n>We present ZR-0, a 2.6 billion parameter end-to-end VLA model that uses dense Embodied Chain-of-Reason (ECoT) supervision to align cross-embodiment representations.
  arXiv  Detail & Related papers  (2026-06-29T16:48:48Z)
• Factorized Latent Dynamics for Video JEPA: An Empirical Study of Auxiliary Objectives [0.0]
  Joint-Embedding Predictive Architectures (JEPA) are a promising framework for self-supervised video representation learning.<n>We report a small-scale empirical study of 18 auxiliary objective variants for Video-JEPA across two pretraining regimes.
  arXiv  Detail & Related papers  (2026-05-16T21:30:51Z)
• HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents [71.09493646932046]
  We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents.<n>The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning.<n>Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro.
  arXiv  Detail & Related papers  (2026-04-08T17:59:48Z)
• Adaptive Enhancement and Dual-Pooling Sequential Attention for Lightweight Underwater Object Detection with YOLOv10 [0.0]
  This manuscript introduces a streamlined yet robust framework for underwater object detection, grounded in the YOLOv10 architecture.<n>The proposed method integrates a Multi-Stage Adaptive Enhancement module to improve image quality and a Dual-Pooling Sequential Attention mechanism to strengthen multi-scale feature representation.
  arXiv  Detail & Related papers  (2026-03-04T07:39:57Z)
• BFA++: Hierarchical Best-Feature-Aware Token Prune for Multi-View Vision Language Action Model [44.72361174037017]
  Vision-Language-Action (VLA) models have achieved significant breakthroughs by leveraging Large Vision Language Models (VLMs) to jointly interpret instructions and visual inputs.<n>The substantial increase in visual tokens, particularly from multi-view inputs, poses serious challenges to real-time robotic manipulation.<n>We propose BFA++, a dynamic token pruning framework designed specifically for VLA models.
  arXiv  Detail & Related papers  (2026-02-24T05:31:52Z)
• Universal Pose Pretraining for Generalizable Vision-Language-Action Policies [83.39008378156647]
  Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency.<n>We propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors.<n>Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion alignment.
  arXiv  Detail & Related papers  (2026-02-23T11:00:08Z)
• Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning [82.39668822222386]
  Vision token pruning has proven to be an effective acceleration technique for the efficient Vision Language Model (VLM)<n>We propose $textNwa$, a two-stage token pruning framework that enables efficient feature aggregation while maintaining spatial integrity.<n>Experiments demonstrate that $textNwa$ achieves SOTA performance on multiple VQA benchmarks (from 94% to 95%) and yields substantial improvements on visual grounding tasks (from 7% to 47%)
  arXiv  Detail & Related papers  (2026-02-03T00:51:03Z)
• Light-Weight Cross-Modal Enhancement Method with Benchmark Construction for UAV-based Open-Vocabulary Object Detection [6.443926939309045]
  We propose a complete UAV-oriented solution that combines both dataset construction and model innovation.<n>First, we design a refined UAV-Label Engine, which efficiently resolves annotation redundancy, inconsistency, and ambiguity.<n>Second, we introduce the Cross-Attention Gated Enhancement (CAGE) module, a lightweight dual-path fusion design that integrates cross-attention, adaptive gating, and global FiLM modulation for robust textvision alignment.
  arXiv  Detail & Related papers  (2025-09-07T10:59:02Z)
• EmbRACE-3K: Embodied Reasoning and Action in Complex Environments [48.32142591866083]
  EmRACE-3K is a dataset of over 3,000 language-guided tasks constructed using Unreal Engine and the UnrealCV-Zoo framework.<n>We establish a benchmark to evaluate the embodied reasoning capabilities of VLMs across three key dimensions: Exploration, Dynamic Spatial-Semantic Reasoning, and Multi-stage Goal Execution.<n>In zero-shot settings, all models achieve success rates below 20%, underscoring the challenge posed by our benchmark and the current limitations of VLMs in interactive environments.
  arXiv  Detail & Related papers  (2025-07-14T17:59:46Z)
• EgoPoseFormer: A Simple Baseline for Stereo Egocentric 3D Human Pose Estimation [15.590340765703893]
  We present EgoPoseFormer, a transformer-based model for stereo egocentric human pose estimation. Our approach overcomes the main challenge of overcoming joint invisibility caused by self-occlusion or a limited field of view (FOV) of head-mounted cameras. We evaluate our method on the stereo UnrealEgo dataset and show it significantly outperforms previous approaches.
  arXiv  Detail & Related papers  (2024-03-26T20:02:48Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• VOLO: Vision Outlooker for Visual Recognition [148.12522298731807]
  Vision transformers (ViTs) have shown great potential of self-attention based models in ImageNet classification. We introduce a novel outlook attention and present a simple and general architecture, termed Vision Outlooker (VOLO) Unlike self-attention that focuses on global dependency modeling at a coarse level, the outlook attention efficiently encodes finer-level features and contexts into tokens. Experiments show that our VOLO achieves 87.1% top-1 accuracy on ImageNet-1K classification, which is the first model exceeding 87% accuracy on this competitive benchmark.
  arXiv  Detail & Related papers  (2021-06-24T15:46:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.