Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification

• URL: http://arxiv.org/abs/2501.16811v1
• Date: Tue, 28 Jan 2025 09:29:13 GMT
• Title: Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification
• Authors: Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu,
• Abstract summary: This paper proposes a new effective and efficient plug-and-play backbone for video-based person re-identification (ReID) We find that different frames in a ReID video often exhibit small differences and contain many similar regions due to the relatively slight movements of human beings. We introduce a patch selection mechanism to reduce computational cost by choosing only the crucial and non-repetitive patches for feature extraction.
• Score: 32.86287519276783
• License:
• Abstract: This paper proposes a new effective and efficient plug-and-play backbone for video-based person re-identification (ReID). Conventional video-based ReID methods typically use CNN or transformer backbones to extract deep features for every position in every sampled video frame. Here, we argue that this exhaustive feature extraction could be unnecessary, since we find that different frames in a ReID video often exhibit small differences and contain many similar regions due to the relatively slight movements of human beings. Inspired by this, a more selective, efficient paradigm is explored in this paper. Specifically, we introduce a patch selection mechanism to reduce computational cost by choosing only the crucial and non-repetitive patches for feature extraction. Additionally, we present a novel network structure that generates and utilizes pseudo frame global context to address the issue of incomplete views resulting from sparse inputs. By incorporating these new designs, our backbone can achieve both high performance and low computational cost. Extensive experiments on multiple datasets show that our approach reduces the computational cost by 74\% compared to ViT-B and 28\% compared to ResNet50, while the accuracy is on par with ViT-B and outperforms ResNet50 significantly.

Related papers

• Sample Less, Learn More: Efficient Action Recognition via Frame Feature Restoration [59.6021678234829]
  We propose a novel method to restore the intermediate features for two sparsely sampled and adjacent video frames. With the integration of our method, the efficiency of three commonly used baselines has been improved by over 50%, with a mere 0.5% reduction in recognition accuracy.
  arXiv  Detail & Related papers  (2023-07-27T13:52:42Z)
• NSNet: Non-saliency Suppression Sampler for Efficient Video Recognition [89.84188594758588]
  A novel Non-saliency Suppression Network (NSNet) is proposed to suppress the responses of non-salient frames. NSNet achieves the state-of-the-art accuracy-efficiency trade-off and presents a significantly faster (2.44.3x) practical inference speed than state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-07-21T09:41:22Z)
• Efficient Meta-Tuning for Content-aware Neural Video Delivery [40.3731358963689]
  We present Efficient Meta-Tuning (EMT) to reduce the computational cost. EMT adapts a meta-learned model to the first chunk of the input video. We propose a novel sampling strategy to extract the most challenging patches from video frames.
  arXiv  Detail & Related papers  (2022-07-20T06:47:10Z)
• Multi-encoder Network for Parameter Reduction of a Kernel-based Interpolation Architecture [10.08097582267397]
  Convolutional neural networks (CNNs) have been at the forefront of the recent advances in this field. Many of these networks require a lot of parameters, with more parameters meaning a heavier burden. This paper presents a method for parameter reduction for a popular flow-less kernel-based network.
  arXiv  Detail & Related papers  (2022-05-13T16:02:55Z)
• OCSampler: Compressing Videos to One Clip with Single-step Sampling [82.0417131211353]
  We propose a framework named OCSampler to explore a compact yet effective video representation with one short clip. Our basic motivation is that the efficient video recognition task lies in processing a whole sequence at once rather than picking up frames sequentially.
  arXiv  Detail & Related papers  (2022-01-12T09:50:38Z)
• Adaptive Focus for Efficient Video Recognition [29.615394426035074]
  We propose a reinforcement learning based approach for efficient spatially adaptive video recognition (AdaFocus) A light-weighted ConvNet is first adopted to quickly process the full video sequence, whose features are used by a recurrent policy network to localize the most task-relevant regions. During offline inference, once the informative patch sequence has been generated, the bulk of computation can be done in parallel, and is efficient on modern GPU devices.
  arXiv  Detail & Related papers  (2021-05-07T13:24:47Z)
• VA-RED$^2$: Video Adaptive Redundancy Reduction [64.75692128294175]
  We present a redundancy reduction framework, VA-RED$2$, which is input-dependent. We learn the adaptive policy jointly with the network weights in a differentiable way with a shared-weight mechanism. Our framework achieves $20% - 40%$ reduction in computation (FLOPs) when compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-15T22:57:52Z)
• Deep Space-Time Video Upsampling Networks [47.62807427163614]
  Video super-resolution (VSR) and frame (FI) are traditional computer vision problems. We propose an end-to-end framework for the space-time video upsampling by efficiently merging VSR and FI into a joint framework. Results show better results both quantitatively and qualitatively, while reducing the time (x7 faster) and the number of parameters (30%) compared to baselines.
  arXiv  Detail & Related papers  (2020-04-06T07:04:21Z)
• Scene-Adaptive Video Frame Interpolation via Meta-Learning [54.87696619177496]
  We propose to adapt the model to each video by making use of additional information that is readily available at test time. We obtain significant performance gains with only a single gradient update without any additional parameters.
  arXiv  Detail & Related papers  (2020-04-02T02:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.