TRIM: A Self-Supervised Video Summarization Framework Maximizing Temporal Relative Information and Representativeness

• URL: http://arxiv.org/abs/2506.20588v1
• Date: Wed, 25 Jun 2025 16:27:38 GMT
• Title: TRIM: A Self-Supervised Video Summarization Framework Maximizing Temporal Relative Information and Representativeness
• Authors: Pritam Mishra, Coloma Ballester, Dimosthenis Karatzas,
• Abstract summary: We introduce a self-supervised video summarization model that captures both spatial and temporal dependencies without the overhead of attention, RNNs, or transformers.<n>Our framework integrates a novel set of Markov process-driven loss metrics and a two-stage self supervised learning paradigm that ensures both performance and efficiency.
• Score: 9.374702244811303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing ubiquity of video content and the corresponding demand for efficient access to meaningful information have elevated video summarization and video highlights as a vital research area. However, many state-of-the-art methods depend heavily either on supervised annotations or on attention-based models, which are computationally expensive and brittle in the face of distribution shifts that hinder cross-domain applicability across datasets. We introduce a pioneering self-supervised video summarization model that captures both spatial and temporal dependencies without the overhead of attention, RNNs, or transformers. Our framework integrates a novel set of Markov process-driven loss metrics and a two-stage self supervised learning paradigm that ensures both performance and efficiency. Our approach achieves state-of-the-art performance on the SUMME and TVSUM datasets, outperforming all existing unsupervised methods. It also rivals the best supervised models, demonstrating the potential for efficient, annotation-free architectures. This paves the way for more generalizable video summarization techniques and challenges the prevailing reliance on complex architectures.

Related papers

• COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition [3.271109623410664]
  We propose COMODO, a cross-modal self-supervised distillation framework that transfers rich semantic knowledge from the video modality to the IMU modality without requiring labeled annotations.<n>Our approach enables the IMU encoder to inherit rich semantic information from video while preserving its efficiency for real-world applications.
  arXiv  Detail & Related papers  (2025-03-10T12:43:51Z)
• Enhancing Video Summarization with Context Awareness [9.861215740353247]
  Video summarization automatically generate concise summaries by selecting techniques, shots, or segments that capture the video's essence. Despite the importance of video summarization, there is a lack of diverse and representative datasets. We propose an unsupervised approach that leverages video data structure and information for generating informative summaries.
  arXiv  Detail & Related papers  (2024-04-06T09:08:34Z)
• Scaling Up Video Summarization Pretraining with Large Language Models [73.74662411006426]
  We introduce an automated and scalable pipeline for generating a large-scale video summarization dataset. We analyze the limitations of existing approaches and propose a new video summarization model that effectively addresses them. Our work also presents a new benchmark dataset that contains 1200 long videos each with high-quality summaries annotated by professionals.
  arXiv  Detail & Related papers  (2024-04-04T11:59:06Z)
• Conditional Modeling Based Automatic Video Summarization [70.96973928590958]
  The aim of video summarization is to shorten videos automatically while retaining the key information necessary to convey the overall story. Video summarization methods rely on visual factors, such as visual consecutiveness and diversity, which may not be sufficient to fully understand the content of the video. A new approach to video summarization is proposed based on insights gained from how humans create ground truth video summaries.
  arXiv  Detail & Related papers  (2023-11-20T20:24:45Z)
• Self-Attention Based Generative Adversarial Networks For Unsupervised Video Summarization [78.2700757742992]
  We build on a popular method where a Generative Adversarial Network (GAN) is trained to create representative summaries. We propose the SUM-GAN-AED model that uses a self-attention mechanism for frame selection, combined with LSTMs for encoding and decoding.
  arXiv  Detail & Related papers  (2023-07-16T19:56:13Z)
• Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning [54.67880602409801]
  In this paper, we study the problem of pre-training world models with abundant in-the-wild videos for efficient learning of visual control tasks. We introduce Contextualized World Models (ContextWM) that explicitly separate context and dynamics modeling. Our experiments show that in-the-wild video pre-training equipped with ContextWM can significantly improve the sample efficiency of model-based reinforcement learning.
  arXiv  Detail & Related papers  (2023-05-29T14:29:12Z)
• An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement [132.60976158877608]
  We propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate-temporal-information across frames and reduces the need for high complexity networks.
  arXiv  Detail & Related papers  (2020-12-24T00:03:29Z)
• Exploring global diverse attention via pairwise temporal relation for video summarization [84.28263235895798]
  We propose an efficient convolutional neural network architecture for video SUMmarization via Global Diverse Attention. The proposed models can be run in parallel with significantly less computational costs.
  arXiv  Detail & Related papers  (2020-09-23T06:29:09Z)
• Transforming Multi-Concept Attention into Video Summarization [36.85535624026879]
  We propose a novel attention-based framework for video summarization with complex video data. Our model can be applied to both labeled and unlabeled data, making our method preferable to real-world applications.
  arXiv  Detail & Related papers  (2020-06-02T06:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.