Temporal Rate Reduction Clustering for Human Motion Segmentation

• URL: http://arxiv.org/abs/2506.21249v2
• Date: Sat, 09 Aug 2025 07:00:56 GMT
• Title: Temporal Rate Reduction Clustering for Human Motion Segmentation
• Abstract summary: We propose Temporal Rate Reduction Clustering ($textTR2textC$), which jointly learns structured representations and affinity to segment the sequences of frames in video.<n>We conduct extensive experiments on five benchmark HMS datasets and achieve state-of-the-art performances with different feature extractors.
• Score: 2.902719887963343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human Motion Segmentation (HMS), which aims to partition videos into non-overlapping human motions, has attracted increasing research attention recently. Existing approaches for HMS are mainly dominated by subspace clustering methods, which are grounded on the assumption that high-dimensional temporal data align with a Union-of-Subspaces (UoS) distribution. However, the frames in video capturing complex human motions with cluttered backgrounds may not align well with the UoS distribution. In this paper, we propose a novel approach for HMS, named Temporal Rate Reduction Clustering ($\text{TR}^2\text{C}$), which jointly learns structured representations and affinity to segment the sequences of frames in video. Specifically, the structured representations learned by $\text{TR}^2\text{C}$ enjoy temporally consistency and are aligned well with a UoS structure, which is favorable for addressing the HMS task. We conduct extensive experiments on five benchmark HMS datasets and achieve state-of-the-art performances with different feature extractors. The code is available at: https://github.com/mengxianghan123/TR2C.

Related papers

• Jointly Learning Structured Representations and Stabilized Affinity for Human Motion Segmentation [7.350724521347576]
  Human Motion Clustering (HMS) aims to partition a video into non-overlapping segments corresponding to different human motions.<n>For videos in the real world, raw frame-level features often violate the Unionof-Subspaces assumption and yield unsatisfactory segmentation performance.<n>We propose Temporal Deep Self-expressive subspace Clustering ( TDSC) which jointly learns temporally consistent structured representations and affinity for accurate and robust HMS.
  arXiv  Detail & Related papers  (2026-05-07T06:48:53Z)
• Temporal Visual Semantics-Induced Human Motion Understanding with Large Language Models [22.529947693320864]
  Unsupervised human motion segmentation can be effectively achieved using subspace clustering techniques.<n>This paper explores the use of temporal vision semantics (TVS) derived from human motion sequences to enhance subspace clustering performance.<n> Experimental results demonstrate that the proposed method outperforms existing state-of-the-art approaches on four benchmark human motion datasets.
  arXiv  Detail & Related papers  (2025-12-24T03:11:12Z)
• MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learning [26.58473269689558]
  Video Moment Retrieval (MR) and Highlight Detection (HD) aim to pinpoint specific moments and assess clip-wise relevance based on the text query.<n>We propose the Motion-Semantics DETR (MS-DETR), a framework that captures rich motion-semantics features through unified learning for MR/HD tasks.<n>Our method outperforms existing state-of-the-art models by a margin.
  arXiv  Detail & Related papers  (2025-07-16T09:18:18Z)
• Scalable Context-Preserving Model-Aware Deep Clustering for Hyperspectral Images [51.95768218975529]
  Subspace clustering has become widely adopted for the unsupervised analysis of hyperspectral images (HSIs)<n>Recent model-aware deep subspace clustering methods often use a two-stage framework, involving the calculation of a self-representation matrix with complexity of O(n2), followed by spectral clustering.<n>We propose a scalable, context-preserving deep clustering method based on basis representation, which jointly captures local and non-local structures for efficient HSI clustering.
  arXiv  Detail & Related papers  (2025-06-12T16:43:09Z)
• Temporally Consistent Referring Video Object Segmentation with Hybrid Memory [98.80249255577304]
  We propose an end-to-end R-VOS paradigm that explicitly models temporal consistency alongside the referring segmentation. Features of frames with automatically generated high-quality reference masks are propagated to segment remaining frames. Extensive experiments demonstrate that our approach enhances temporal consistency by a significant margin.
  arXiv  Detail & Related papers  (2024-03-28T13:32:49Z)
• OMH: Structured Sparsity via Optimally Matched Hierarchy for Unsupervised Semantic Segmentation [69.37484603556307]
  Un Semantic segmenting (USS) involves segmenting images without relying on predefined labels. We introduce a novel approach called Optimally Matched Hierarchy (OMH) to simultaneously address the above issues. Our OMH yields better unsupervised segmentation performance compared to existing USS methods.
  arXiv  Detail & Related papers  (2024-03-11T09:46:41Z)
• Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling [96.64607294592062]
  Video Semantic Label Roleing (VidSRL) aims to detect salient events from given videos. Recent endeavors have put forth methods for VidSRL, but they can be subject to two key drawbacks.
  arXiv  Detail & Related papers  (2023-08-09T17:20:14Z)
• Hierarchical Spatio-Temporal Representation Learning for Gait Recognition [6.877671230651998]
  Gait recognition is a biometric technique that identifies individuals by their unique walking styles. We propose a hierarchical-temporal representation learning framework for extracting gait features from coarse to fine. Our method outperforms the state-of-the-art while maintaining a reasonable balance between model accuracy and complexity.
  arXiv  Detail & Related papers  (2023-07-19T09:30:00Z)
• Local-Global Temporal Difference Learning for Satellite Video Super-Resolution [53.03380679343968]
  We propose to exploit the well-defined temporal difference for efficient and effective temporal compensation.<n>To fully utilize the local and global temporal information within frames, we systematically modeled the short-term and long-term temporal discrepancies.<n> Rigorous objective and subjective evaluations conducted across five mainstream video satellites demonstrate that our method performs favorably against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-04-10T07:04:40Z)
• TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering [27.52568444236988]
  We propose an unsupervised approach for learning action classes from untrimmed video sequences. In particular, we propose a temporal embedding network that combines relative time prediction, feature reconstruction, and sequence-to-sequence learning. Based on the identified clusters, we decode the video into coherent temporal segments that correspond to semantically meaningful action classes.
  arXiv  Detail & Related papers  (2023-03-09T10:46:23Z)
• Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences [29.376328807860993]
  We propose a Partial Spatio-Temporal Learning (PSTL) framework to exploit the local relationship between different skeleton joints and video frames. Our method achieves state-of-the-art performance on NTURGB+D 60, NTURGBMM+D 120 and PKU-D under various downstream tasks.
  arXiv  Detail & Related papers  (2023-02-17T17:35:05Z)
• Temporally-Consistent Surface Reconstruction using Metrically-Consistent Atlases [131.50372468579067]
  We propose a method for unsupervised reconstruction of a temporally-consistent sequence of surfaces from a sequence of time-evolving point clouds. We represent the reconstructed surfaces as atlases computed by a neural network, which enables us to establish correspondences between frames. Our approach outperforms state-of-the-art ones on several challenging datasets.
  arXiv  Detail & Related papers  (2021-11-12T17:48:25Z)
• Video Is Graph: Structured Graph Module for Video Action Recognition [34.918667614077805]
  We transform a video sequence into a graph to obtain direct long-term dependencies among temporal frames. In particular, SGM divides the neighbors of each node into several temporal regions so as to extract global structural information. The reported performance and analysis demonstrate that SGM can achieve outstanding precision with less computational complexity.
  arXiv  Detail & Related papers  (2021-10-12T11:27:29Z)
• Spatial-Spectral Clustering with Anchor Graph for Hyperspectral Image [88.60285937702304]
  This paper proposes a novel unsupervised approach called spatial-spectral clustering with anchor graph (SSCAG) for HSI data clustering. The proposed SSCAG is competitive against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-04-24T08:09:27Z)
• Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos [78.45050529204701]
  We propose a novel framework to pursue discriminative and robust representation by modeling cross-scale spatial-temporal correlation. CTL utilizes a CNN backbone and a key-points estimator to extract semantic local features from human body. It explores a context-reinforced topology to construct multi-scale graphs by considering both global contextual information and physical connections of human body.
  arXiv  Detail & Related papers  (2021-04-15T14:32:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.