Spatial-Temporal Alignment Network for Action Recognition

• URL: http://arxiv.org/abs/2308.09897v1
• Date: Sat, 19 Aug 2023 03:31:57 GMT
• Title: Spatial-Temporal Alignment Network for Action Recognition
• Authors: Jinhui Ye and Junwei Liang
• Abstract summary: This paper studies introducing viewpoint invariant feature representations in existing action recognition architecture. We propose a novel Spatial-Temporal Alignment Network (STAN), which explicitly learns geometric invariant representations for action recognition. We test our STAN model on widely-used datasets like UCF101 and HMDB51.
• Score: 5.2170672727035345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper studies introducing viewpoint invariant feature representations in existing action recognition architecture. Despite significant progress in action recognition, efficiently handling geometric variations in large-scale datasets remains challenging. To tackle this problem, we propose a novel Spatial-Temporal Alignment Network (STAN), which explicitly learns geometric invariant representations for action recognition. Notably, the STAN model is light-weighted and generic, which could be plugged into existing action recognition models (e.g., MViTv2) with a low extra computational cost. We test our STAN model on widely-used datasets like UCF101 and HMDB51. The experimental results show that the STAN model can consistently improve the state-of-the-art models in action recognition tasks in trained-from-scratch settings.

Related papers

• Idempotent Unsupervised Representation Learning for Skeleton-Based Action Recognition [13.593511876719367]
  We propose a novel skeleton-based idempotent generative model (IGM) for unsupervised representation learning. Our experiments on benchmark datasets, NTU RGB+D and PKUMMD, demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2024-10-27T06:29:04Z)
• Generalizable Implicit Neural Representation As a Universal Spatiotemporal Traffic Data Learner [46.866240648471894]
  Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. We present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales.
  arXiv  Detail & Related papers  (2024-06-13T02:03:22Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• Spatiotemporal Implicit Neural Representation as a Generalized Traffic Data Learner [46.866240648471894]
  Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. We present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales.
  arXiv  Detail & Related papers  (2024-05-06T06:23:06Z)
• SOAR: Advancements in Small Body Object Detection for Aerial Imagery Using State Space Models and Programmable Gradients [0.8873228457453465]
  Small object detection in aerial imagery presents significant challenges in computer vision. Traditional methods using transformer-based models often face limitations stemming from the lack of specialized databases. This paper introduces two innovative approaches that significantly enhance detection and segmentation capabilities for small aerial objects.
  arXiv  Detail & Related papers  (2024-05-02T19:47:08Z)
• D$^2$ST-Adapter: Disentangled-and-Deformable Spatio-Temporal Adapter for Few-shot Action Recognition [60.84084172829169]
  Adapting large pre-trained image models to few-shot action recognition has proven to be an effective strategy for learning robust feature extractors. We present the Disentangled-and-Deformable Spatio-Temporal Adapter (D$2$ST-Adapter), which is a novel tuning framework well-suited for few-shot action recognition.
  arXiv  Detail & Related papers  (2023-12-03T15:40:10Z)
• Latent Variable Representation for Reinforcement Learning [131.03944557979725]
  It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning. We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle. In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
  arXiv  Detail & Related papers  (2022-12-17T00:26:31Z)
• ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation [135.10594078615952]
  We introduce ACID, an action-conditional visual dynamics model for volumetric deformable objects. A benchmark contains over 17,000 action trajectories with six types of plush toys and 78 variants. Our model achieves the best performance in geometry, correspondence, and dynamics predictions.
  arXiv  Detail & Related papers  (2022-03-14T04:56:55Z)
• Progressive Self-Guided Loss for Salient Object Detection [102.35488902433896]
  We present a progressive self-guided loss function to facilitate deep learning-based salient object detection in images. Our framework takes advantage of adaptively aggregated multi-scale features to locate and detect salient objects effectively.
  arXiv  Detail & Related papers  (2021-01-07T07:33:38Z)
• Spatial-Temporal Alignment Network for Action Recognition and Detection [80.19235282200697]
  This paper studies how to introduce viewpoint-invariant feature representations that can help action recognition and detection. We propose a novel Spatial-Temporal Alignment Network (STAN) that aims to learn geometric invariant representations for action recognition and action detection. We test our STAN model extensively on AVA, Kinetics-400, AVA-Kinetics, Charades, and Charades-Ego datasets.
  arXiv  Detail & Related papers  (2020-12-04T06:23:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.