IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning

• URL: http://arxiv.org/abs/2605.01668v1
• Date: Sun, 03 May 2026 01:45:58 GMT
• Title: IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
• Abstract summary: IMPACT-Scribe is a correction-driven framework for dense labeling.<n>It combines uncertainty-aware boundary supervision, local proposal modeling, cost-aware query planning, structured propagation, and correction-driven adaptation.<n> Experiments and a human study show that this closed-loop design improves labeling quality per effort, enhances boundary accuracy, and fosters better humanmachine interaction over time.
• Score: 75.72434806468884
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correction is treated as an isolated edit, limiting reuse of information on annotator uncertainty and model reliability. We introduce IMPACT-Scribe, a correction-driven framework for dense labeling that uses each correction to improve future human-machine collaboration. IMPACT-Scribe combines uncertainty-aware boundary scribble supervision, local proposal modeling, cost-aware query planning, structured propagation, and correction-driven adaptation. Experiments and a human study show that this closed-loop design improves labeling quality per effort, enhances boundary accuracy, and fosters better human-machine interaction over time. The code will be made publicly available at https://github.com/BanzQians/IMPACT_AS.

Related papers

• IMPACT: Attention Is the Interaction Map for Scalable Interaction-Aware World Model Training [18.292611164084537]
  We develop IMPACT, a scalable Interaction-aware framework with Prior-guided Attention and Targeting.<n> IMPACT consistently outperforms the corresponding MSE-trained baselines, improving interaction fidelity, physical plausibility, and visual quality.
  arXiv  Detail & Related papers  (2026-08-31T18:00:36Z)
• Improving Temporal Action Segmentation via Constraint-Aware Decoding [18.2371734918413]
  Temporal action segmentation (TAS) divides untrimmed videos into labeled action segments.<n>We propose a lightweight, constraint-based refinement framework that enhances TAS predictions.<n>Our approach improves both fully and semi-supervised TAS models by correcting structural prediction errors.
  arXiv  Detail & Related papers  (2026-05-11T07:55:02Z)
• IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction [72.51952455865155]
  We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video.<n> IMPACT-HOI frames this task as the incremental resolution of a partially specified, onset-anchored event state.<n>A user study with 9 participants shows a 13.5% reduction in manual annotation actions, a 46.67% event match rate, and zero confirmed-field violations.
  arXiv  Detail & Related papers  (2026-05-03T01:37:40Z)
• Generating Realistic, Protocol-Compliant Maritime Radio Dialogues using Self-Instruct and Low-Rank Adaptation [0.0]
  Human factors account for over 58% of recorded incidents in Europe between 2014 and 2023.<n>VHF radio communications are still prone to noise, interference, linguistic variability, and the absence of real-time transcription.<n>This study introduces a compliance aware Self-Instruct methodology for generating realistic maritime radio dialogues that conform to the IMO's SMCP.
  arXiv  Detail & Related papers  (2026-02-16T09:09:27Z)
• From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models [42.553888395227766]
  We propose a lightweight, training-free framework for vision-language-action (VLA) models.<n>VLA-SCT operates as a self-correcting control loop, combining data-driven action refinement with conditional logic for termination.<n>Our method achieves consistent improvements across all datasets in the LIBERO benchmark.
  arXiv  Detail & Related papers  (2026-02-02T08:44:40Z)
• REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegration [11.462236606266567]
  Post-training for large language models (LLMs) is constrained by the high cost of acquiring new knowledge or correcting errors.<n>This work introduces a robust framework for developing reliable, scalable, and continually evolving LLMs.
  arXiv  Detail & Related papers  (2025-10-02T10:35:39Z)
• AutoLayout: Closed-Loop Layout Synthesis via Slow-Fast Collaborative Reasoning [102.71841660031065]
  Auto is a fully automated method that integrates a closed-loop self-validation process within a dual-system framework.<n>The effectiveness of Auto was validated across 8 distinct scenarios, where it demonstrated a significant 10.1% improvement over SOTA methods.
  arXiv  Detail & Related papers  (2025-07-06T08:35:22Z)
• AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification [25.27444694706659]
  We present AskToAct, which exploits the structural mapping between queries and their tool invocation solutions.<n>By systematically removing key parameters from queries while retaining them as ground truth, we enable automated construction of high-quality training data.<n>Our framework exhibits robust performance across different model architectures and successfully generalizes to entirely unseen APIs without additional training.
  arXiv  Detail & Related papers  (2025-03-03T12:55:49Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment [93.09267863425492]
  We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable. We construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures.
  arXiv  Detail & Related papers  (2022-04-07T17:59:32Z)
• Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding [53.377028000325424]
  We propose an Iterative Alignment Network (IA-Net) for temporal sentence grounding task. We pad multi-modal features with learnable parameters to alleviate the nowhere-to-attend problem of non-matched frame-word pairs. We also devise a calibration module following each attention module to refine the alignment knowledge.
  arXiv  Detail & Related papers  (2021-09-14T02:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.