Improving Temporal Action Segmentation via Constraint-Aware Decoding

• URL: http://arxiv.org/abs/2605.10149v1
• Date: Mon, 11 May 2026 07:55:02 GMT
• Title: Improving Temporal Action Segmentation via Constraint-Aware Decoding
• Abstract summary: Temporal action segmentation (TAS) divides untrimmed videos into labeled action segments.<n>We propose a lightweight, constraint-based refinement framework that enhances TAS predictions.<n>Our approach improves both fully and semi-supervised TAS models by correcting structural prediction errors.
• Score: 18.2371734918413
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal action segmentation (TAS) divides untrimmed videos into labeled action segments. While fully supervised methods have advanced the field, challenges such as action variability, ambiguous boundaries, and high annotation costs remain, especially in new or low-resource domains. Grammar-based approaches improve segmentation with structural priors but rely on complex parsing limiting scalability. In this work, we propose a lightweight, constraint-based refinement framework that enhances TAS predictions by integrating statistical structural priors such as transition confidence, action boundary sets, and per-class duration, that can be directly extracted from annotated data. These constraints are integrated into a modified Viterbi decoding algorithm, allowing inference-time refinement without retraining or added model complexity. Our approach improves both fully and semi-supervised TAS models by correcting structural prediction errors while maintaining high efficiency. Code is available at https://github.com/LUNAProject22/CAD

Related papers

• IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning [75.72434806468884]
  IMPACT-Scribe is a correction-driven framework for dense labeling.<n>It combines uncertainty-aware boundary supervision, local proposal modeling, cost-aware query planning, structured propagation, and correction-driven adaptation.<n> Experiments and a human study show that this closed-loop design improves labeling quality per effort, enhances boundary accuracy, and fosters better humanmachine interaction over time.
  arXiv  Detail & Related papers  (2026-05-03T01:45:58Z)
• Boundary-Centric Active Learning for Temporal Action Segmentation [0.34785805439031336]
  We introduce B-ACT, a clip-budgeted active learning framework that explicitly allocates supervision to high-leverage boundary regions.<n>B-ACT operates in a hierarchical two-stage loop: (i) it ranks and queries unlabeled videos using predictive uncertainty, and (ii) within each selected video, it selects the top-$K$ boundaries.
  arXiv  Detail & Related papers  (2026-04-16T15:50:21Z)
• Combining Boundary Supervision and Segment-Level Regularization for Fine-Grained Action Segmentation [20.7405249370601]
  We present a lightweight dual-loss training framework that improves fine-grained segmentation quality with only one additional output channel and two auxiliary loss terms.<n>Our approach combines a boundary-regression loss that promotes accurate temporal localization via a single-channel boundary prediction and a CDF-based segment-level regularization loss.<n>Across three benchmark datasets, the proposed method improves segment-level consistency and boundary quality, yielding higher F1 and Edit scores across three different models.
  arXiv  Detail & Related papers  (2026-04-02T10:15:33Z)
• Decomposing and Composing: Towards Efficient Vision-Language Continual Learning via Rank-1 Expert Pool in a Single LoRA [50.97792275353563]
  We introduce a novel framework that restructures a single Low-Rank Adaptation (LoRA) module as a decomposable Rank-1 Expert Pool.<n>Our method learns to dynamically compose a sparse, task-specific update by selecting from this expert pool, guided by the semantics of the [Guided] token.
  arXiv  Detail & Related papers  (2026-01-30T10:54:51Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.<n>We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• HELPNet: Hierarchical Perturbations Consistency and Entropy-guided Ensemble for Scribble Supervised Medical Image Segmentation [4.034121387622003]
  We propose HELPNet, a novel scribble-based weakly supervised segmentation framework.<n>HELPNet integrates three modules to bridge the gap between annotation efficiency and segmentation performance.<n>HELPNet significantly outperforms state-of-the-art methods for scribble-based weakly supervised segmentation.
  arXiv  Detail & Related papers  (2024-12-25T01:52:01Z)
• Dynamic Integration of Task-Specific Adapters for Class Incremental Learning [32.124078616314144]
  Non-exemplar class Incremental Learning (NECIL) enables models to continuously acquire new classes without retraining from scratch and storing old task exemplars.<n>We propose a novel framework called Dynamic Integration of task-specific Adapters (DIA), which comprises two key components: Task-Specific Adapter Integration (TSAI) and Patch-Level Model Alignment.
  arXiv  Detail & Related papers  (2024-09-23T13:01:33Z)
• Faster Diffusion Action Segmentation [9.868244939496678]
  Temporal Action Classification (TAS) is an essential task in video analysis, aiming to segment and classify continuous frames into distinct action segments. Recent advances in diffusion models have demonstrated substantial success in TAS tasks due to their stable training process and high-quality generation capabilities. We propose EffiDiffAct, an efficient and high-performance TAS algorithm.
  arXiv  Detail & Related papers  (2024-08-04T13:23:18Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation [74.8791451327354]
  We propose a simple yet effective semi-supervised learning framework for semantic segmentation. A set of simple design and training techniques can collectively improve the performance of semi-supervised semantic segmentation significantly. Our method achieves state-of-the-art results in the semi-supervised settings on the Cityscapes and Pascal VOC datasets.
  arXiv  Detail & Related papers  (2021-04-15T06:01:39Z)
• Boundary-sensitive Pre-training for Temporal Localization in Videos [124.40788524169668]
  We investigate model pre-training for temporal localization by introducing a novel boundary-sensitive pretext ( BSP) task. With the synthesized boundaries, BSP can be simply conducted via classifying the boundary types. Extensive experiments show that the proposed BSP is superior and complementary to the existing action classification based pre-training counterpart.
  arXiv  Detail & Related papers  (2020-11-21T17:46:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.