SpeedNet: Learning the Speediness in Videos
- URL: http://arxiv.org/abs/2004.06130v2
- Date: Sun, 26 Jul 2020 14:33:04 GMT
- Title: SpeedNet: Learning the Speediness in Videos
- Abstract summary: SpeedNet is a novel deep network trained to detect if a video is playing at normal rate, or if it is sped up.
We show how this single, binary classification network can be used to detect arbitrary rates of speediness of objects.
- Score: 54.798526633714474
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We wish to automatically predict the "speediness" of moving objects in
videos---whether they move faster, at, or slower than their "natural" speed.
The core component in our approach is SpeedNet---a novel deep network trained
to detect if a video is playing at normal rate, or if it is sped up. SpeedNet
is trained on a large corpus of natural videos in a self-supervised manner,
without requiring any manual annotations. We show how this single, binary
classification network can be used to detect arbitrary rates of speediness of
objects. We demonstrate prediction results by SpeedNet on a wide range of
videos containing complex natural motions, and examine the visual cues it
utilizes for making those predictions. Importantly, we show that through
predicting the speed of videos, the model learns a powerful and meaningful
space-time representation that goes beyond simple motion cues. We demonstrate
how those learned features can boost the performance of self-supervised action
recognition, and can be used for video retrieval. Furthermore, we also apply
SpeedNet for generating time-varying, adaptive video speedups, which can allow
viewers to watch videos faster, but with less of the jittery, unnatural motions
typical to videos that are sped up uniformly.
Related papers
- Seeing Fast and Slow: Learning the Flow of Time in Videos [58.71893154112877]
We study time as a learnable visual concept and develop models for reasoning about and manipulating the flow of time in videos.<n>We curate the largest slow-motion video dataset to date from noisy in-the-wild sources.<n>These learned temporal reasoning models enable us to curate the largest slow-motion video dataset to date from noisy in-the-wild sources.
arXiv Detail & Related papers (2026-04-23T17:59:57Z) - Adaptive Caching for Faster Video Generation with Diffusion Transformers [52.73348147077075]
Diffusion Transformers (DiTs) rely on larger models and heavier attention mechanisms, resulting in slower inference speeds.
We introduce a training-free method to accelerate video DiTs, termed Adaptive Caching (AdaCache)
We also introduce a Motion Regularization (MoReg) scheme to utilize video information within AdaCache, controlling the compute allocation based on motion content.
arXiv Detail & Related papers (2024-11-04T18:59:44Z) - Arena: A Patch-of-Interest ViT Inference Acceleration System for Edge-Assisted Video Analytics [18.042752812489276]
We introduce an end-to-end edge-assisted video inference acceleration system based on Vision Transformer (ViT)
Our findings reveal that Arena can boost inference speeds by up to 1.58(times) and 1.82(times) on average while consuming only 47% and 31% of the bandwidth, respectively, all with high inference accuracy.
arXiv Detail & Related papers (2024-04-14T13:14:13Z) - NSNet: Non-saliency Suppression Sampler for Efficient Video Recognition [89.84188594758588]
A novel Non-saliency Suppression Network (NSNet) is proposed to suppress the responses of non-salient frames.
NSNet achieves the state-of-the-art accuracy-efficiency trade-off and presents a significantly faster (2.44.3x) practical inference speed than state-of-the-art methods.
arXiv Detail & Related papers (2022-07-21T09:41:22Z) - Video-ReTime: Learning Temporally Varying Speediness for Time Remapping [12.139222986297263]
We train a neural network through self-supervision to recognize and accurately localize changes in the video playback speed.
We demonstrate that this model can detect playback speed variations more accurately while also being orders of magnitude more efficient than prior approaches.
arXiv Detail & Related papers (2022-05-11T16:27:47Z) - MoViNets: Mobile Video Networks for Efficient Video Recognition [52.49314494202433]
3D convolutional neural networks (CNNs) are accurate at video recognition but require large computation and memory budgets.
We propose a three-step approach to improve computational efficiency while substantially reducing the peak memory usage of 3D CNNs.
arXiv Detail & Related papers (2021-03-21T23:06:38Z) - RSPNet: Relative Speed Perception for Unsupervised Video Representation
Learning [100.76672109782815]
We study unsupervised video representation learning that seeks to learn both motion and appearance features from unlabeled video only.
It is difficult to construct a suitable self-supervised task to well model both motion and appearance features.
We propose a new way to perceive the playback speed and exploit the relative speed between two video clips as labels.
arXiv Detail & Related papers (2020-10-27T16:42:50Z) - Self-supervised Video Representation Learning by Pace Prediction [48.029602040786685]
This paper addresses the problem of self-supervised video representation learning from a new perspective -- by video pace prediction.
It stems from the observation that human visual system is sensitive to video pace.
We randomly sample training clips in different paces and ask a neural network to identify the pace for each video clip.
arXiv Detail & Related papers (2020-08-13T12:40:24Z) - Self-Supervised Visual Learning by Variable Playback Speeds Prediction
of a Video [23.478555947694108]
We propose a self-supervised visual learning method by predicting the variable playback speeds of a video.
We learn the meta-temporal visual variations in the video by leveraging the variations in the visual appearance according to playback speeds.
We also propose a new layer dependable temporal group normalization method that can be applied to 3D convolutional networks.
arXiv Detail & Related papers (2020-03-05T15:01:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.