PRISM: Predictive Recomposition via Semantic Latent Decomposition for View-invariant Video Representation Learning

• URL: http://arxiv.org/abs/2608.30388v1
• Date: Mon, 31 Aug 2026 07:43:02 GMT
• Title: PRISM: Predictive Recomposition via Semantic Latent Decomposition for View-invariant Video Representation Learning
• Abstract summary: Cross-view video representation learning aims to capture viewpoint-invariant action semantics.<n>We propose PRISM, that decomposes video into view-invariant and view-variant latents and recompose them under language supervision.<n> PRISM achieves state-of-the-art results on EgoExo4D, EgoExoLearn, AE2, even surpassing in-domain models under zero-shot setting.
• Score: 57.66429303062003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-view video representation learning aims to capture viewpoint-invariant action semantics despite substantial appearance changes across egocentric and exocentric videos. However, existing methods encode each video as a unified embedding, where view-invariant and view-variant semantics inevitably entangle under co-occurrences - a failure mode we show persists even in cross-view methods explicitly trained for view-invariance. Our key insight is that a view-invariant feature is truly disentangled when it can be sufficiently recomposed with an arbitrary view-variant feature while preserving their independent semantics. Building on this, we propose PRISM, that decomposes video into view-invariant and view-variant latents and recompose them under language supervision encouraging clean decomposition of the two streams. PRISM achieves state-of-the-art results on EgoExo4D, EgoExoLearn, AE2, even surpassing in-domain models under zero-shot setting. Code is available at https://github.com/litcoderr/prism.

Related papers

• ID-V2V: Identity-Preserving Video Restylization [16.492389645021028]
  We formalize the challenge of identity-preserving video restylization, which propagates scene, lighting, and style changes specified by an edited video.<n>A key obstacle is the absence of paired training data, as identity-preserving restylized video pairs are rare in real-world settings.<n>We introduce ID-V2V, a video-to-video generative framework integrating complementary control signals.
  arXiv  Detail & Related papers  (2026-07-24T18:12:58Z)
• Learning Activity View-invariance Under Extreme Viewpoint Changes via Curriculum Knowledge Distillation [67.7916602652644]
  Methods for view-invariant learning from video rely on controlled multi-view settings with minimal scene clutter.<n>We introduce a method for learning rich video representations in the presence of such severe view-occlusions.<n>We evaluate our approach on two tasks, outperforming SOTA models on both temporal keystep grounding and fine-grained keystep recognition benchmarks.
  arXiv  Detail & Related papers  (2025-04-07T19:30:30Z)
• Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representations [47.04855334955006]
  We propose a novel masked ego-exo modeling that promotes both causal temporal dynamics and cross-view alignment.<n>We highlight the importance of capturing the compositional nature of human actions as a basis for robust cross-view understanding.
  arXiv  Detail & Related papers  (2025-03-25T14:33:32Z)
• POV: Prompt-Oriented View-Agnostic Learning for Egocentric Hand-Object Interaction in the Multi-View World [59.545114016224254]
  Humans are good at translating third-person observations of hand-object interactions into an egocentric view. We propose a Prompt-Oriented View-agnostic learning framework, which enables this view adaptation with few egocentric videos.
  arXiv  Detail & Related papers  (2024-03-09T09:54:44Z)
• Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos [25.910110689486952]
  We propose a novel benchmark for cross-view knowledge transfer of dense video captioning.<n>We adapt models from web instructional videos with exocentric views to an egocentric view.<n>Our experiments confirm the effectiveness of overcoming the view change problem and knowledge transfer to egocentric views.
  arXiv  Detail & Related papers  (2023-11-28T02:51:13Z)
• Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment [71.16699226211504]
  We propose to learn fine-grained action features that are invariant to the viewpoints by aligning egocentric and exocentric videos in time. To this end, we propose AE2, a self-supervised embedding approach with two key designs. For evaluation, we establish a benchmark for fine-grained video understanding in the ego-exo context.
  arXiv  Detail & Related papers  (2023-06-08T19:54:08Z)
• ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints [47.54827916387143]
  We propose ViewCLR, that learns self-supervised video representation invariant to camera viewpoint changes. We introduce a view-generator that can be considered as a learnable augmentation for any self-supervised pre-text tasks.
  arXiv  Detail & Related papers  (2021-12-07T18:58:29Z)
• Time-Equivariant Contrastive Video Representation Learning [47.50766781135863]
  We introduce a novel self-supervised contrastive learning method to learn representations from unlabelled videos. Our experiments show that time-equivariant representations achieve state-of-the-art results in video retrieval and action recognition benchmarks.
  arXiv  Detail & Related papers  (2021-12-07T10:45:43Z)
• JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargeting [53.28477676794658]
  unsupervised motion in videos has seen substantial advancements through the use of deep neural networks. We introduce JOKR - a JOint Keypoint Representation that handles both the source and target videos, without requiring any object prior or data collection. We evaluate our method both qualitatively and quantitatively, and demonstrate that our method handles various cross-domain scenarios, such as different animals, different flowers, and humans.
  arXiv  Detail & Related papers  (2021-06-17T17:32:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.