Fugu-MT: arxivの論文翻訳

P2P-Former: Bridging the Supervision Gap Between Point and Proposal in Temporal Action Localization

Authors Yunchuan Ma, Laiyun Qing, Sai Xu, Guorong Li, Qingming Huang
Affiliations University of the Chinese Academy of Sciences
Categories Method / Temporal Action Localization / Bridging supervision gap, Application / Action Recognition / Temporal action proposals, Task / Supervised Learning / Point-level to full supervision transition
License CC BY 4.0

Abstract Overview

This paper proposes P2P-Former, a Transformer-based framework for point-supervised temporal action localization that aims to bridge the gap between sparse point annotations and proposal-level learning. The method addresses two stated challenges: generating reliable pseudo proposals from point supervision and injecting point-level cues into a proposal-based detector. To do this, it combines point-aware proposal generation with one-to-one matching, a point-guided semantic enhancement module built from clustered point features, and a point-to-proposal constraint loss. Experiments on THUMOS14 and ActivityNet-1.3 show consistent improvements over prior point-supervised methods, with ablations supporting the contribution of each component.

Novelty

The distinctive aspect of the work is its explicit reformulation of point-supervised temporal action localization as a point-to-proposal learning problem rather than relying mainly on snippet-level classification and thresholding. Its main novelties are the point-aware reliable proposal generation with Hungarian matching, prototype-based semantic enhancement from point annotations, and a loss that constrains predicted proposals to include and align with annotated points.

Results

On THUMOS14, the method reports 65.8 average mAP over IoU 0.1:0.1:0.7, exceeding prior point-supervised results in the table, including HR-Pro at 60.3 and QROT at 52.1 for the 0.3:0.1:0.7 average comparison shown. On ActivityNet-1.3, it reports 31.1 average mAP over IoU 0.5:0.05:0.95, above the listed point-supervised baselines such as QROT at 27.7. Ablation results show that combining all three proposed components improves the THUMOS14 average mAP over IoU 0.1:0.1:0.7 from 62.6 to 65.8.

Key Points

  1. P2P-Former generates pseudo ground-truth proposals by re-scoring candidate proposals with point-aware importance and selecting one proposal per point via Hungarian matching.
  2. The framework enriches snippet features using category prototypes clustered from point-annotated features and injects them through Transformer-based interactions.
  3. The full model achieves the best reported point-supervised performance in the provided tables on both THUMOS14 and ActivityNet-1.3, and ablations indicate that PPG, PSE, and PPCL are complementary.

References

This page was created using generative AI such as GPT-5, Claude Opus 4, Gemini 3, Gemini 3.1 Flash Image, and their higher-end successor versions. No guarantee can be made regarding its contents.