Deep video representation learning: a survey

• URL: http://arxiv.org/abs/2405.06574v1
• Date: Fri, 10 May 2024 16:20:11 GMT
• Title: Deep video representation learning: a survey
• Authors: Elham Ravanbakhsh, Yongqing Liang, J. Ramanujam, Xin Li,
• Abstract summary: We recent sequential feature learning methods for visual data and compare their pros and cons for general video analysis. Building effective features for videos is a fundamental problem in computer vision tasks involving video analysis and understanding.
• Score: 4.9589745881431435
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper provides a review on representation learning for videos. We classify recent spatiotemporal feature learning methods for sequential visual data and compare their pros and cons for general video analysis. Building effective features for videos is a fundamental problem in computer vision tasks involving video analysis and understanding. Existing features can be generally categorized into spatial and temporal features. Their effectiveness under variations of illumination, occlusion, view and background are discussed. Finally, we discuss the remaining challenges in existing deep video representation learning studies.

Related papers

• Knowledge-enhanced Multi-perspective Video Representation Learning for Scene Recognition [33.800842679024164]
  We address the problem of video scene recognition, whose goal is to learn a high-level video representation to classify scenes in videos. Most existing works identify scenes for videos only from visual or textual information in a temporal perspective. We propose a novel two-stream framework to model video representations from multiple perspectives.
  arXiv  Detail & Related papers  (2024-01-09T04:37:10Z)
• Contextual Explainable Video Representation:\\Human Perception-based Understanding [10.172332586182792]
  We discuss approaches that incorporate the human perception process into modeling actors, objects, and the environment. We choose video paragraph captioning and temporal action detection to illustrate the effectiveness of human perception based-contextual representation in video understanding.
  arXiv  Detail & Related papers  (2022-12-12T19:29:07Z)
• TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency [133.75876535332003]
  We focus on summarizing instructional videos, an under-explored area of video summarization. Existing video summarization datasets rely on manual frame-level annotations. We propose an instructional video summarization network that combines a context-aware temporal video encoder and a segment scoring transformer.
  arXiv  Detail & Related papers  (2022-08-14T04:07:40Z)
• Self-Supervised Learning for Videos: A Survey [70.37277191524755]
  Self-supervised learning has shown promise in both image and video domains. In this survey, we provide a review of existing approaches on self-supervised learning focusing on the video domain.
  arXiv  Detail & Related papers  (2022-06-18T00:26:52Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• A Survey on Deep Learning Technique for Video Segmentation [147.0767454918527]
  Video segmentation plays a critical role in a broad range of practical applications. Deep learning based approaches have been dedicated to video segmentation and delivered compelling performance.
  arXiv  Detail & Related papers  (2021-07-02T15:51:07Z)
• Highlight Timestamp Detection Model for Comedy Videos via Multimodal Sentiment Analysis [1.6181085766811525]
  We propose a multimodal structure to obtain state-of-the-art performance in this field. We select several benchmarks for multimodal video understanding and apply the most suitable model to find the best performance.
  arXiv  Detail & Related papers  (2021-05-28T08:39:19Z)
• Audiovisual Highlight Detection in Videos [78.26206014711552]
  We present results from two experiments: efficacy study of single features on the task, and an ablation study where we leave one feature out at a time. For the video summarization task, our results indicate that the visual features carry most information, and including audiovisual features improves over visual-only information. Results indicate that we can transfer knowledge from the video summarization task to a model trained specifically for the task of highlight detection.
  arXiv  Detail & Related papers  (2021-02-11T02:24:00Z)
• Video Summarization Using Deep Neural Networks: A Survey [72.98424352264904]
  Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization.
  arXiv  Detail & Related papers  (2021-01-15T11:41:29Z)
• Classification of Important Segments in Educational Videos using Multimodal Features [10.175871202841346]
  We propose a multimodal neural architecture that utilizes state-of-the-art audio, visual and textual features. Our experiments investigate the impact of visual and temporal information, as well as the combination of multimodal features on importance prediction.
  arXiv  Detail & Related papers  (2020-10-26T14:40:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.