Learning Probabilistic Embeddings for Unsupervised Action Segmentation

• URL: http://arxiv.org/abs/2607.05263v1
• Date: Mon, 06 Jul 2026 16:10:25 GMT
• Title: Learning Probabilistic Embeddings for Unsupervised Action Segmentation
• Abstract summary: We study the problem of unsupervised temporal action segmentation for long, untrimmed videos.<n>We learn a probabilistic embedding for frame representations.<n>Compared to baselines with deterministic embeddings, our approach improves MoF up to 20.7% and F1-score up to 19.0%.
• Score: 28.558361258499527
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper concerns the problem of unsupervised temporal action segmentation for long, untrimmed videos. Recent successful approaches follow a joint representation learning and clustering paradigm, where optimal transport (OT) is adopted to produce pseudo labels for learning frame representations. These approaches alternate between estimating pseudo labels using OT and optimizing the parameters with gradient descent during training, where OT is used for obtaining the final temporal action segmentation. A major limitation of these works is that they learn a deterministic embedding for frame representations. The iterative procedure between learning deterministic embeddings based on pseudo labels and estimating pseudo labels from the learned embedding can thus get quickly stuck in a local optimum. As an alternative, we thus propose to learn a probabilistic embedding for frame representations. The embeddings are modeled by Gaussian distributions and we sample from the distributions before estimating the pseudo labels. We evaluate our approach on several challenging temporal action segmentation datasets and achieve results comparable to, and in some cases, better than the state of the art. Compared to baselines with deterministic embeddings, our approach improves MoF up to 20.7\% and F1-score up to 19.0\%. Our code is available at https://github.com/derkbreeze/PEOT.

Related papers

• Amortized Variational Inference for Partial-Label Learning: A Probabilistic Approach to Label Disambiguation [2.7214777196418645]
  Partial-label learning trains classifiers when each instance is associated with a set of candidate labels, only one of which is correct.<n>We introduce a novel framework that directly approximates the posterior distribution over true labels using amortized variational inference.<n>Our method employs neural networks to predict variational parameters from input data, enabling efficient inference.
  arXiv  Detail & Related papers  (2025-10-24T09:54:23Z)
• Active Learning via Classifier Impact and Greedy Selection for Interactive Image Retrieval [4.699825956909531]
  Active Learning (AL) is a user-interactive approach aimed at reducing annotation costs by selecting the most crucial examples to label.<n>We introduce a novel batch-mode Active Learning framework named GAL (Greedy Active Learning) that better copes with this application.
  arXiv  Detail & Related papers  (2024-12-03T09:27:46Z)
• Exploiting Counter-Examples for Active Learning with Partial labels [45.665996618836516]
  This paper studies a new problem, emphactive learning with partial labels (ALPL) In this setting, an oracle annotates the query samples with partial labels, relaxing the oracle from the demanding accurate labeling process. We propose a simple but effective WorseNet to directly learn from this pattern.
  arXiv  Detail & Related papers  (2023-07-14T15:41:53Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Learning with Neighbor Consistency for Noisy Labels [69.83857578836769]
  We present a method for learning from noisy labels that leverages similarities between training examples in feature space. We evaluate our method on datasets evaluating both synthetic (CIFAR-10, CIFAR-100) and realistic (mini-WebVision, Clothing1M, mini-ImageNet-Red) noise.
  arXiv  Detail & Related papers  (2022-02-04T15:46:27Z)
• Dash: Semi-Supervised Learning with Dynamic Thresholding [72.74339790209531]
  We propose a semi-supervised learning (SSL) approach that uses unlabeled examples to train models. Our proposed approach, Dash, enjoys its adaptivity in terms of unlabeled data selection.
  arXiv  Detail & Related papers  (2021-09-01T23:52:29Z)
• Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training [38.81973113564937]
  Self-training is a standard approach to semi-supervised learning where the learner's own predictions on unlabeled data are used as supervision during training. In this paper, we reinterpret this label assignment problem as an optimal transportation problem between examples and classes. We demonstrate the effectiveness of our algorithm on the CIFAR-10, CIFAR-100, and SVHN datasets in comparison with FixMatch, a state-of-the-art self-training algorithm.
  arXiv  Detail & Related papers  (2021-02-17T08:23:15Z)
• Coping with Label Shift via Distributionally Robust Optimisation [72.80971421083937]
  We propose a model that minimises an objective based on distributionally robust optimisation (DRO) We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective.
  arXiv  Detail & Related papers  (2020-10-23T08:33:04Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)
• Progressive Identification of True Labels for Partial-Label Learning [112.94467491335611]
  Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label. Most existing methods elaborately designed as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data. This paper proposes a novel framework of classifier with flexibility on the model and optimization algorithm.
  arXiv  Detail & Related papers  (2020-02-19T08:35:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.