BID: Boundary-Interior Decoding for Unsupervised Temporal Action Localization Pre-Trainin

• URL: http://arxiv.org/abs/2403.07354v1
• Date: Tue, 12 Mar 2024 06:23:45 GMT
• Title: BID: Boundary-Interior Decoding for Unsupervised Temporal Action Localization Pre-Trainin
• Abstract summary: We propose the first unsupervised pre-training framework that partitions a skeleton-based motion sequence into semantically meaningful pre-action segments. By fine-tuning our pre-training network with a small number of annotated data, we show results out-performing SOTA methods by a large margin.
• Score: 13.273908640951252
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Skeleton-based motion representations are robust for action localization and understanding for their invariance to perspective, lighting, and occlusion, compared with images. Yet, they are often ambiguous and incomplete when taken out of context, even for human annotators. As infants discern gestures before associating them with words, actions can be conceptualized before being grounded with labels. Therefore, we propose the first unsupervised pre-training framework, Boundary-Interior Decoding (BID), that partitions a skeleton-based motion sequence into discovered semantically meaningful pre-action segments. By fine-tuning our pre-training network with a small number of annotated data, we show results out-performing SOTA methods by a large margin.

Related papers

• Unsupervised Point Cloud Registration via Training-Time Semantic Guidance [46.92635414824713]
  CAESAR is a teacher-student framework guided by an off-the-shelf 3D segmentation model exclusively during training.<n>Building on this, a train-only Semantic-Geometric Label Mining performs lightweight, batch-specific teacher refinement and mines reliable pseudo-labels.<n>Experiments on KITTI and nuScenes demonstrate state-of-the-art performance, with pronounced gains on the challenging nuScenes benchmark.
  arXiv  Detail & Related papers  (2026-09-14T08:49:09Z)
• TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving [1.5327485092552822]
  Birds-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception.<n>Current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow.<n>We introduce Text-Guided Representation for Instance Prediction (TGRIP), a novel framework that bridges this gap by injecting rich semantic priors into the instance prediction loop.
  arXiv  Detail & Related papers  (2026-07-06T08:47:56Z)
• A Pre-Training Analogue of Grokking in Language Models: Tracing Delayed Grammatical Generalization [11.118638230247951]
  Grokking, the phenomenon in which neural networks generalize long after fitting their training data, has been studied in supervised settings on many epochs.<n>We propose an exposure-based framework that enables the study of grokking-like dynamics during LLM pre-training.
  arXiv  Detail & Related papers  (2026-05-29T18:04:52Z)
• A Progressive Training Strategy for Vision-Language Models to Counteract Spatio-Temporal Hallucinations in Embodied Reasoning [49.61652671596548]
  "Multi-image hallucination reasoning", where a massive performance drop between forward and temporal queries reveals a dependence on superficial shortcuts instead of genuine understanding.<n>To mitigate this, we develop a new dataset: Chain-of-temporal building on detailed reasoning into Chain steps and definitive judgments.<n>Our experiments demonstrate that this approach not only improves accuracy but also the forward-backward performance gap from over 70% to only 6.53%.
  arXiv  Detail & Related papers  (2026-04-12T07:48:44Z)
• Provable Ordering and Continuity in Vision-Language Pretraining for Generalizable Embodied Agents [39.95793203302782]
  Pre-training vision-language representations on human action videos has emerged as a promising approach to reduce reliance on large-scale expert demonstrations for training embodied agents. We propose Action Temporal Coherence Learning (AcTOL) to learn ordered and continuous vision-language representations without rigid goal-based constraint.
  arXiv  Detail & Related papers  (2025-02-03T10:16:49Z)
• FinePseudo: Improving Pseudo-Labelling through Temporal-Alignablity for Semi-Supervised Fine-Grained Action Recognition [57.17966905865054]
  Real-life applications of action recognition often require a fine-grained understanding of subtle movements. Existing semi-supervised action recognition has mainly focused on coarse-grained action recognition. We propose an Alignability-Verification-based Metric learning technique to effectively discriminate between fine-grained action pairs.
  arXiv  Detail & Related papers  (2024-09-02T20:08:06Z)
• Causal Unsupervised Semantic Segmentation [60.178274138753174]
  Unsupervised semantic segmentation aims to achieve high-quality semantic grouping without human-labeled annotations. We propose a novel framework, CAusal Unsupervised Semantic sEgmentation (CAUSE), which leverages insights from causal inference.
  arXiv  Detail & Related papers  (2023-10-11T10:54:44Z)
• Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation [100.81837601210597]
  We propose Concept Curation (CoCu) to bridge the gap between visual and textual semantics in pre-training data. CoCu achieves superb zero-shot transfer performance and greatly boosts language-supervised segmentation baseline by a large margin.
  arXiv  Detail & Related papers  (2023-09-24T00:05:39Z)
• ZScribbleSeg: Zen and the Art of Scribble Supervised Medical Image Segmentation [16.188681108101196]
  We propose to utilize solely scribble annotations for weakly supervised segmentation. Existing solutions mainly leverage selective losses computed solely on annotated areas. We introduce regularization terms to encode the spatial relationship and shape prior. We integrate the efficient scribble supervision with the prior into a unified framework, denoted as ZScribbleSeg.
  arXiv  Detail & Related papers  (2023-01-12T09:00:40Z)
• SegTAD: Precise Temporal Action Detection via Semantic Segmentation [65.01826091117746]
  We formulate the task of temporal action detection in a novel perspective of semantic segmentation. Owing to the 1-dimensional property of TAD, we are able to convert the coarse-grained detection annotations to fine-grained semantic segmentation annotations for free. We propose an end-to-end framework SegTAD composed of a 1D semantic segmentation network (1D-SSN) and a proposal detection network (PDN)
  arXiv  Detail & Related papers  (2022-03-03T06:52:13Z)
• Boundary-aware Self-supervised Learning for Video Scene Segmentation [20.713635723315527]
  Video scene segmentation is a task of temporally localizing scene boundaries in a video. We introduce three novel boundary-aware pretext tasks: Shot-Scene Matching, Contextual Group Matching and Pseudo-boundary Prediction. We achieve the new state-of-the-art on the MovieNet-SSeg benchmark.
  arXiv  Detail & Related papers  (2022-01-14T02:14:07Z)
• Towards Tokenized Human Dynamics Representation [41.75534387530019]
  We study how to segment and cluster videos into recurring temporal patterns in a self-supervised way. We evaluate the frame-wise representation learning step by Kendall's Tau and the lexicon building step by normalized mutual information and language entropy. On the AIST++ and PKU-MMD datasets, actons bring significant performance improvements compared to several baselines.
  arXiv  Detail & Related papers  (2021-11-22T18:59:58Z)
• Weakly Supervised Temporal Adjacent Network for Language Grounding [96.09453060585497]
  We introduce a novel weakly supervised temporal adjacent network (WSTAN) for temporal language grounding. WSTAN learns cross-modal semantic alignment by exploiting temporal adjacent network in a multiple instance learning (MIL) paradigm. An additional self-discriminating loss is devised on both the MIL branch and the complementary branch, aiming to enhance semantic discrimination by self-supervising.
  arXiv  Detail & Related papers  (2021-06-30T15:42:08Z)
• Self-Supervision by Prediction for Object Discovery in Videos [62.87145010885044]
  In this paper, we use the prediction task as self-supervision and build a novel object-centric model for image sequence representation. Our framework can be trained without the help of any manual annotation or pretrained network. Initial experiments confirm that the proposed pipeline is a promising step towards object-centric video prediction.
  arXiv  Detail & Related papers  (2021-03-09T19:14:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.