Temporal vs. Spatial: Comparing DINOv3 and V-JEPA2 Feature Representations for Video Action Analysis

• URL: http://arxiv.org/abs/2509.21595v1
• Date: Thu, 25 Sep 2025 21:05:07 GMT
• Title: Temporal vs. Spatial: Comparing DINOv3 and V-JEPA2 Feature Representations for Video Action Analysis
• Abstract summary: DINOv3 processes frames independently through spatial feature extraction, and V-JEPA2 employs joint temporal modeling across video sequences.<n>We evaluate both approaches on the UCF Sports dataset, examining feature quality through multiple dimensions including classification accuracy, clustering performance, intra-class consistency, and inter-class discrimination.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study presents a comprehensive comparative analysis of two prominent self-supervised learning architectures for video action recognition: DINOv3, which processes frames independently through spatial feature extraction, and V-JEPA2, which employs joint temporal modeling across video sequences. We evaluate both approaches on the UCF Sports dataset, examining feature quality through multiple dimensions including classification accuracy, clustering performance, intra-class consistency, and inter-class discrimination. Our analysis reveals fundamental architectural trade-offs: DINOv3 achieves superior clustering performance (Silhouette score: 0.31 vs 0.21) and demonstrates exceptional discrimination capability (6.16x separation ratio) particularly for pose-identifiable actions, while V-JEPA2 exhibits consistent reliability across all action types with significantly lower performance variance (0.094 vs 0.288). Through action-specific evaluation, we identify that DINOv3's spatial processing architecture excels at static pose recognition but shows degraded performance on motion-dependent actions, whereas V-JEPA2's temporal modeling provides balanced representation quality across diverse action categories. These findings contribute to the understanding of architectural design choices in video analysis systems and provide empirical guidance for selecting appropriate feature extraction methods based on task requirements and reliability constraints.

Related papers

• CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval [54.15776146365823]
  Composed Image Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text.<n>We propose CSMCIR, a unified representation framework that achieves efficient query-target alignment through three synergistic components.
  arXiv  Detail & Related papers  (2026-01-07T09:21:38Z)
• Feature Hallucination for Self-supervised Action Recognition [37.20267786858476]
  We propose a deep translational action recognition framework that enhances recognition accuracy by jointly predicting action concepts and auxiliary features from RGB video frames.<n>Our framework achieves state-of-the-art performance on multiple benchmarks, including Kinetics-400, Kinetics-600, and Something-Something V2, demonstrating its effectiveness in capturing fine-grained action dynamics.
  arXiv  Detail & Related papers  (2025-06-25T11:50:23Z)
• Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation [8.912201177914858]
  Saliency-Motion guided Trunk-Collateral Network (SMTC-Net)<n>We propose a novel Trunk-Collateral structure for motion-appearance video object segmentation (UVOS)<n>SMTC-Net achieved state-of-the-art performance on three UVOS datasets.
  arXiv  Detail & Related papers  (2025-04-08T11:02:14Z)
• Deep Content Understanding Toward Entity and Aspect Target Sentiment Analysis on Foundation Models [0.8602553195689513]
  Entity-Aspect Sentiment Triplet Extraction (EASTE) is a novel Aspect-Based Sentiment Analysis task. Our research aims to achieve high performance on the EASTE task and investigates the impact of model size, type, and adaptation techniques on task performance. Ultimately, we provide detailed insights and achieving state-of-the-art results in complex sentiment analysis.
  arXiv  Detail & Related papers  (2024-07-04T16:48:14Z)
• Distilling Aggregated Knowledge for Weakly-Supervised Video Anomaly Detection [11.250490586786878]
  Video anomaly detection aims to develop automated models capable of identifying abnormal events in surveillance videos.<n>We show that distilling knowledge from aggregated representations of multiple backbones into a single-backbone Student model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-06-05T00:44:42Z)
• Aligning in a Compact Space: Contrastive Knowledge Distillation between Heterogeneous Architectures [4.119589507611071]
  We propose a Low-Frequency Components-based Contrastive Knowledge Distillation (LFCC) framework that significantly enhances the performance of feature-based distillation. Specifically, we designe a set of multi-scale low-pass filters to extract the low-frequency components of intermediate features from both the teacher and student models. We show that LFCC achieves superior performance on the challenging benchmarks of ImageNet-1K and CIFAR-100.
  arXiv  Detail & Related papers  (2024-05-28T18:44:42Z)
• Collaboratively Self-supervised Video Representation Learning for Action Recognition [54.92120002380786]
  We design a Collaboratively Self-supervised Video Representation learning framework specific to action recognition.<n>Our method achieves state-of-the-art performance on multiple popular video datasets.
  arXiv  Detail & Related papers  (2024-01-15T10:42:04Z)
• CAST: Cross-Attention in Space and Time for Video Action Recognition [8.785207228156098]
  We propose a novel two-stream architecture called Cross-Attention in Space and Time (CAST) CAST achieves a balanced spatial-temporal understanding of videos using only balanced input. Our proposed mechanism enables spatial and temporal expert models to exchange information and make synergistic predictions.
  arXiv  Detail & Related papers  (2023-11-30T18:58:51Z)
• Multi-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimation [49.56131393810713]
  We present an SE(3) equivariant architecture and a training strategy to tackle this task in an unsupervised manner. Our method excels in both model performance and computational efficiency, with only 0.25M parameters and 0.92G FLOPs.
  arXiv  Detail & Related papers  (2023-06-08T22:55:32Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Spatio-Temporal Representation Factorization for Video-based Person Re-Identification [55.01276167336187]
  We propose Spatio-Temporal Representation Factorization module (STRF) for re-ID. STRF is a flexible new computational unit that can be used in conjunction with most existing 3D convolutional neural network architectures for re-ID. We empirically show that STRF improves performance of various existing baseline architectures while demonstrating new state-of-the-art results.
  arXiv  Detail & Related papers  (2021-07-25T19:29:37Z)
• Unified Graph Structured Models for Video Understanding [93.72081456202672]
  We propose a message passing graph neural network that explicitly models relational-temporal relations. We show how our method is able to more effectively model relationships between relevant entities in the scene.
  arXiv  Detail & Related papers  (2021-03-29T14:37:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.