Improving Viewpoint-Invariance and Temporal Consistency for Action Detection

• URL: http://arxiv.org/abs/2605.22695v1
• Date: Thu, 21 May 2026 16:38:21 GMT
• Title: Improving Viewpoint-Invariance and Temporal Consistency for Action Detection
• Abstract summary: This paper introduces a novel two-stage action detection approach.<n>In the first stage, we extract motion features from augmented virtual viewpoints.<n>The second stage introduces a view-invariant, multi-scale temporal encoder.
• Score: 2.0019258650495493
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Viewpoint change invariance and action temporal consistency are critical aspects for the effective deployment of human action detection of untrimmed videos. Existing appearance-based video detection methods often struggle with limited viewpoint diversity during training, while motion-based detection approaches frequently fail to model fine-grained temporal relationships across consecutive motion windows. This paper introduces a novel two-stage action detection approach designed to improve both view-invariance and global temporal coherence properties. In the first stage, we extract motion features from augmented virtual viewpoints, solely used at training. Then, the second stage introduces a new view-invariant, multi-scale temporal encoder based on selective state-space sequence modelling to aggregate information across viewpoints and time scales. Experiments on PKU-MMD and BABEL benchmarks demonstrate that this approach significantly outperforms state-of-the-art methods in all considered splits. Code and trained models are available at: https://icb-vision-ai.github.io/HydraView-TAD

Related papers

• Adaptive Multi-Granularity Temporal Modeling for Weakly Supervised Video Anomaly Detection [1.2072875728151518]
  weakly supervised video anomaly detection (WSVAD) has emerged as a critical research frontier.<n>Most existing approaches formulate WSVAD within a Multiple Instance Learning (MIL) framework that relies on rigid, hand-crafted temporal priors to supervise anomaly scoring.<n>We propose an adaptive temporal modeling framework for WSVAD that explicitly accounts for variations in video dynamics across multiple temporal granularities.
  arXiv  Detail & Related papers  (2026-09-04T12:27:43Z)
• Action Quality Assessment via Hierarchical Pose-guided Multi-stage Contrastive Regression [46.25518274714238]
  Action Assessment (AQA) aims at automatic and fair evaluation of athletic performance.<n>Current methods focus on segmenting video into fixed frames, which disrupts the temporal continuity of sub-actions.<n>We propose a novel action quality assessment method through hierarchically pose-guided multi-stage contrastive regression.
  arXiv  Detail & Related papers  (2025-01-07T10:20:16Z)
• Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection [19.643936110623653]
  Video Anomaly Detection (VAD) aims to identify abnormalities within a specific context and timeframe. Recent deep learning-based VAD models have shown promising results by generating high-resolution frames. We propose a self-supervised learning approach for VAD through an inter-patch relationship prediction task.
  arXiv  Detail & Related papers  (2024-03-28T03:07:16Z)
• Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling [19.683714649646603]
  Weakly-supervised action localization aims to recognize and localize action instancese in untrimmed videos with only video-level labels. Most existing models rely on multiple instance learning(MIL), where predictions of unlabeled instances are supervised by classifying labeled bags. We propose a novel attention-based hierarchically-structured latent model to learn the temporal variations of feature semantics.
  arXiv  Detail & Related papers  (2023-08-19T08:45:49Z)
• Gait Recognition in the Wild with Multi-hop Temporal Switch [81.35245014397759]
  gait recognition in the wild is a more practical problem that has attracted the attention of the community of multimedia and computer vision. This paper presents a novel multi-hop temporal switch method to achieve effective temporal modeling of gait patterns in real-world scenes.
  arXiv  Detail & Related papers  (2022-09-01T10:46:09Z)
• Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localization [87.47977407022492]
  This paper argues that learning by contextually comparing sequence-to-sequence distinctions offers an essential inductive bias in weakly-supervised action localization. Under a differentiable dynamic programming formulation, two complementary contrastive objectives are designed, including Fine-grained Sequence Distance (FSD) contrasting and Longest Common Subsequence (LCS) contrasting. Our method achieves state-of-the-art performance on two popular benchmarks.
  arXiv  Detail & Related papers  (2022-03-31T05:13:50Z)
• Implicit Motion Handling for Video Camouflaged Object Detection [60.98467179649398]
  We propose a new video camouflaged object detection (VCOD) framework. It can exploit both short-term and long-term temporal consistency to detect camouflaged objects from video frames.
  arXiv  Detail & Related papers  (2022-03-14T17:55:41Z)
• Slow-Fast Visual Tempo Learning for Video-based Action Recognition [78.3820439082979]
  Action visual tempo characterizes the dynamics and the temporal scale of an action. Previous methods capture the visual tempo either by sampling raw videos with multiple rates, or by hierarchically sampling backbone features. We propose a Temporal Correlation Module (TCM) to extract action visual tempo from low-level backbone features at single-layer remarkably.
  arXiv  Detail & Related papers  (2022-02-24T14:20:04Z)
• Efficient Modelling Across Time of Human Actions and Interactions [92.39082696657874]
  We argue that current fixed-sized-temporal kernels in 3 convolutional neural networks (CNNDs) can be improved to better deal with temporal variations in the input. We study how we can better handle between classes of actions, by enhancing their feature differences over different layers of the architecture. The proposed approaches are evaluated on several benchmark action recognition datasets and show competitive results.
  arXiv  Detail & Related papers  (2021-10-05T15:39:11Z)
• TTAN: Two-Stage Temporal Alignment Network for Few-shot Action Recognition [29.95184808021684]
  Few-shot action recognition aims to recognize novel action classes (query) using just a few samples (support) We devise a novel multi-shot fusion strategy, which takes the misalignment among support samples into consideration. Experiments on benchmark datasets show the potential of the proposed method in achieving state-of-the-art performance for few-shot action recognition.
  arXiv  Detail & Related papers  (2021-07-10T07:22:49Z)
• Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detection [81.32280287658486]
  We propose a novel one-stage method, namely Glance and Gaze Network (GGNet) GGNet adaptively models a set of actionaware points (ActPoints) via glance and gaze steps. We design an actionaware approach that effectively matches each detected interaction with its associated human-object pair.
  arXiv  Detail & Related papers  (2021-04-12T08:01:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.