A Video Summarization Method Using Temporal Interest Detection and Key Frame Prediction

• URL: http://arxiv.org/abs/2109.12581v1
• Date: Sun, 26 Sep 2021 12:15:18 GMT
• Title: A Video Summarization Method Using Temporal Interest Detection and Key Frame Prediction
• Authors: Yubo An and Shenghui Zhao
• Abstract summary: Video summarization is formulated as a combination of sequence labeling and temporal interest detection problem. In our method, we firstly built a flexible universal network frame to simultaneously predicts frame-level importance scores and temporal interest segments. Tests and analysis on two benchmark datasets prove the effectiveness of our method.
• Score: 3.9596068699962323
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, a Video Summarization Method using Temporal Interest Detection and Key Frame Prediction is proposed for supervised video summarization, where video summarization is formulated as a combination of sequence labeling and temporal interest detection problem. In our method, we firstly built a flexible universal network frame to simultaneously predicts frame-level importance scores and temporal interest segments, and then combine the two components with different weights to achieve a more detailed video summarization. Extensive experiments and analysis on two benchmark datasets prove the effectiveness of our method. Specifically, compared with other state-of-the-art methods, its performance is increased by at least 2.6% and 4.2% on TVSum and SumMe respectively.

Related papers

• Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection [19.643936110623653]
  Video Anomaly Detection (VAD) aims to identify abnormalities within a specific context and timeframe. Recent deep learning-based VAD models have shown promising results by generating high-resolution frames. We propose a self-supervised learning approach for VAD through an inter-patch relationship prediction task.
  arXiv  Detail & Related papers  (2024-03-28T03:07:16Z)
• Causal Video Summarizer for Video Exploration [74.27487067877047]
  Causal Video Summarizer (CVS) is proposed to capture the interactive information between the video and query. Based on the evaluation of the existing multi-modal video summarization dataset, experimental results show that the proposed approach is effective.
  arXiv  Detail & Related papers  (2023-07-04T22:52:16Z)
• Dual Prototype Attention for Unsupervised Video Object Segmentation [28.725754274542304]
  Unsupervised video object segmentation (VOS) aims to detect and segment the most salient object in videos. This paper proposes two novel prototype-based attention mechanisms, inter-modality attention (IMA) and inter-frame attention (IFA)
  arXiv  Detail & Related papers  (2022-11-22T06:19:17Z)
• Deep Unsupervised Key Frame Extraction for Efficient Video Classification [63.25852915237032]
  This work presents an unsupervised method to retrieve the key frames, which combines Convolutional Neural Network (CNN) and Temporal Segment Density Peaks Clustering (TSDPC) The proposed TSDPC is a generic and powerful framework and it has two advantages compared with previous works, one is that it can calculate the number of key frames automatically. Furthermore, a Long Short-Term Memory network (LSTM) is added on the top of the CNN to further elevate the performance of classification.
  arXiv  Detail & Related papers  (2022-11-12T20:45:35Z)
• Coarse-Fine Networks for Temporal Activity Detection in Videos [45.03545172714305]
  We introduce 'Co-Fine Networks', a two-stream architecture which benefits from different abstractions of temporal resolution to learn better video representations for long-term motion. We show that our method can outperform the state-of-the-arts for action detection in public datasets with a significantly reduced compute and memory footprint.
  arXiv  Detail & Related papers  (2021-03-01T20:48:01Z)
• Semi-Supervised Action Recognition with Temporal Contrastive Learning [50.08957096801457]
  We learn a two-pathway temporal contrastive model using unlabeled videos at two different speeds. We considerably outperform video extensions of sophisticated state-of-the-art semi-supervised image recognition methods.
  arXiv  Detail & Related papers  (2021-02-04T17:28:35Z)
• Efficient video annotation with visual interpolation and frame selection guidance [0.0]
  We introduce a unified framework for generic video annotation with bounding boxes. We show that our approach reduces actual measured annotation time by 50% compared to commonly used linear methods.
  arXiv  Detail & Related papers  (2020-12-23T09:31:40Z)
• Finding Action Tubes with a Sparse-to-Dense Framework [62.60742627484788]
  We propose a framework that generates action tube proposals from video streams with a single forward pass in a sparse-to-dense manner. We evaluate the efficacy of our model on the UCF101-24, JHMDB-21 and UCFSports benchmark datasets.
  arXiv  Detail & Related papers  (2020-08-30T15:38:44Z)
• Temporal Context Aggregation for Video Retrieval with Contrastive Learning [81.12514007044456]
  We propose TCA, a video representation learning framework that incorporates long-range temporal information between frame-level features. The proposed method shows a significant performance advantage (17% mAP on FIVR-200K) over state-of-the-art methods with video-level features.
  arXiv  Detail & Related papers  (2020-08-04T05:24:20Z)
• Video Super-resolution with Temporal Group Attention [127.21615040695941]
  We propose a novel method that can effectively incorporate temporal information in a hierarchical way. The input sequence is divided into several groups, with each one corresponding to a kind of frame rate. It achieves favorable performance against state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-07-21T04:54:30Z)
• Transforming Multi-Concept Attention into Video Summarization [36.85535624026879]
  We propose a novel attention-based framework for video summarization with complex video data. Our model can be applied to both labeled and unlabeled data, making our method preferable to real-world applications.
  arXiv  Detail & Related papers  (2020-06-02T06:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.