Sign Segmentation with Changepoint-Modulated Pseudo-Labelling
- URL: http://arxiv.org/abs/2104.13817v1
- Date: Wed, 28 Apr 2021 15:05:19 GMT
- Title: Sign Segmentation with Changepoint-Modulated Pseudo-Labelling
- Authors: Katrin Renz, Nicolaj C. Stache, Neil Fox, G\"ul Varol, Samuel Albanie
- Abstract summary: The objective of this work is to find temporal boundaries between signs in continuous sign language.
Motivated by the paucity of annotation available for this task, we propose a simple yet effective algorithm to improve segmentation performance.
- Score: 12.685780222519902
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The objective of this work is to find temporal boundaries between signs in
continuous sign language. Motivated by the paucity of annotation available for
this task, we propose a simple yet effective algorithm to improve segmentation
performance on unlabelled signing footage from a domain of interest. We make
the following contributions: (1) We motivate and introduce the task of
source-free domain adaptation for sign language segmentation, in which labelled
source data is available for an initial training phase, but is not available
during adaptation. (2) We propose the Changepoint-Modulated Pseudo-Labelling
(CMPL) algorithm to leverage cues from abrupt changes in motion-sensitive
feature space to improve pseudo-labelling quality for adaptation. (3) We
showcase the effectiveness of our approach for category-agnostic sign
segmentation, transferring from the BSLCORPUS to the BSL-1K and
RWTH-PHOENIX-Weather 2014 datasets, where we outperform the prior state of the
art.
Related papers
- PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning [34.786268652516355]
Unsupervised domain adaptive segmentation aims to improve the segmentation accuracy of models on target domains without relying on labeled data from those domains.
It seeks to align the feature representations of the source domain (where labeled data is available) and the target domain (where only unlabeled data is present)
arXiv Detail & Related papers (2024-07-24T08:53:29Z) - Semantic Connectivity-Driven Pseudo-labeling for Cross-domain
Segmentation [89.41179071022121]
Self-training is a prevailing approach in cross-domain semantic segmentation.
We propose a novel approach called Semantic Connectivity-driven pseudo-labeling.
This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics.
arXiv Detail & Related papers (2023-12-11T12:29:51Z) - DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation [78.30720731968135]
Unsupervised domain adaptation in semantic segmentation has been raised to alleviate the reliance on expensive pixel-wise annotations.
We propose DecoupleNet that alleviates source domain overfitting and enables the final model to focus more on the segmentation task.
We also put forward Self-Discrimination (SD) and introduce an auxiliary classifier to learn more discriminative target domain features with pseudo labels.
arXiv Detail & Related papers (2022-07-20T15:47:34Z) - Labeling Where Adapting Fails: Cross-Domain Semantic Segmentation with
Point Supervision via Active Selection [81.703478548177]
Training models dedicated to semantic segmentation require a large amount of pixel-wise annotated data.
Unsupervised domain adaptation approaches aim at aligning the feature distributions between the labeled source and the unlabeled target data.
Previous works attempted to include human interactions in this process under the form of sparse single-pixel annotations in the target data.
We propose a new domain adaptation framework for semantic segmentation with annotated points via active selection.
arXiv Detail & Related papers (2022-06-01T01:52:28Z) - LabOR: Labeling Only if Required for Domain Adaptive Semantic
Segmentation [79.96052264984469]
We propose a human-in-the-loop approach to adaptively give scarce labels to points that a UDA model is uncertain about.
We show the advantages of this new framework for domain adaptive semantic segmentation while minimizing human labor costs.
arXiv Detail & Related papers (2021-08-12T07:35:40Z) - Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature
Alignment [32.77436219094282]
SSDAS employs a few labeled target samples as anchors for adaptive and progressive feature alignment between labeled source samples and unlabeled target samples.
In addition, we replace the dissimilar source features by high-confidence target features continuously during the iterative training process.
Extensive experiments show the proposed SSDAS greatly outperforms a number of baselines.
arXiv Detail & Related papers (2021-06-05T09:12:50Z) - Dual-Refinement: Joint Label and Feature Refinement for Unsupervised
Domain Adaptive Person Re-Identification [51.98150752331922]
Unsupervised domain adaptive (UDA) person re-identification (re-ID) is a challenging task due to the missing of labels for the target domain data.
We propose a novel approach, called Dual-Refinement, that jointly refines pseudo labels at the off-line clustering phase and features at the on-line training phase.
Our method outperforms the state-of-the-art methods by a large margin.
arXiv Detail & Related papers (2020-12-26T07:35:35Z) - Sign language segmentation with temporal convolutional networks [25.661006537351547]
Our approach employs 3D convolutional neural network representations with iterative temporal segment refinement to resolve ambiguities between sign boundary cues.
We demonstrate the effectiveness of our approach on the BSLCORPUS, PHOENIX14 and BSL-1K datasets.
arXiv Detail & Related papers (2020-11-25T19:11:48Z) - Domain Adaptive Semantic Segmentation Using Weak Labels [115.16029641181669]
We propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain.
We develop a weak-label classification module to enforce the network to attend to certain categories.
In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting.
arXiv Detail & Related papers (2020-07-30T01:33:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.