Multi-View Correlation Consistency for Semi-Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2208.08437v1
• Date: Wed, 17 Aug 2022 17:59:11 GMT
• Title: Multi-View Correlation Consistency for Semi-Supervised Semantic Segmentation
• Authors: Yunzhong Hou, Stephen Gould, Liang Zheng
• Abstract summary: Semi-supervised semantic segmentation needs rich and robust supervision on unlabeled data. We propose a view-coherent data augmentation strategy that guarantees pixel-pixel correspondence between different views. In a series of semi-supervised settings on two datasets, we report competitive accuracy compared with the state-of-the-art methods.
• Score: 59.34619548026885
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semi-supervised semantic segmentation needs rich and robust supervision on unlabeled data. Consistency learning enforces the same pixel to have similar features in different augmented views, which is a robust signal but neglects relationships with other pixels. In comparison, contrastive learning considers rich pairwise relationships, but it can be a conundrum to assign binary positive-negative supervision signals for pixel pairs. In this paper, we take the best of both worlds and propose multi-view correlation consistency (MVCC) learning: it considers rich pairwise relationships in self-correlation matrices and matches them across views to provide robust supervision. Together with this correlation consistency loss, we propose a view-coherent data augmentation strategy that guarantees pixel-pixel correspondence between different views. In a series of semi-supervised settings on two datasets, we report competitive accuracy compared with the state-of-the-art methods. Notably, on Cityscapes, we achieve 76.8% mIoU with 1/8 labeled data, just 0.6% shy from the fully supervised oracle.

Related papers

• DealMVC: Dual Contrastive Calibration for Multi-view Clustering [78.54355167448614]
  We propose a novel Dual contrastive calibration network for Multi-View Clustering (DealMVC) We first design a fusion mechanism to obtain a global cross-view feature. Then, a global contrastive calibration loss is proposed by aligning the view feature similarity graph and the high-confidence pseudo-label graph. During the training procedure, the interacted cross-view feature is jointly optimized at both local and global levels.
  arXiv  Detail & Related papers  (2023-08-17T14:14:28Z)
• CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation [73.89509052503222]
  This paper presents a simple but performant semi-supervised semantic segmentation approach, called CorrMatch. We observe that the correlation maps not only enable clustering pixels of the same category easily but also contain good shape information. We propose to conduct pixel propagation by modeling the pairwise similarities of pixels to spread the high-confidence pixels and dig out more. Then, we perform region propagation to enhance the pseudo labels with accurate class-agnostic masks extracted from the correlation maps.
  arXiv  Detail & Related papers  (2023-06-07T10:02:29Z)
• BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency [66.8685113725007]
  BiCro aims to estimate soft labels for noisy data pairs to reflect their true correspondence degree. experiments on three popular cross-modal matching datasets demonstrate that BiCro significantly improves the noise-robustness of various matching models.
  arXiv  Detail & Related papers  (2023-03-22T09:33:50Z)
• SimMatch: Semi-supervised Learning with Similarity Matching [43.61802702362675]
  SimMatch is a new semi-supervised learning framework that considers semantic similarity and instance similarity. With 400 epochs of training, SimMatch achieves 67.2%, and 74.4% Top-1 Accuracy with 1% and 10% labeled examples on ImageNet.
  arXiv  Detail & Related papers  (2022-03-14T08:08:48Z)
• ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering [52.491074276133325]
  We propose an augmentation-free graph contrastive learning framework to solve the problem of partial multi-view clustering. The proposed approach elevates instance-level contrastive learning and missing data inference to the cluster-level, effectively mitigating the impact of individual missing data on clustering.
  arXiv  Detail & Related papers  (2022-03-01T02:32:25Z)
• Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learning [53.74240452117145]
  This paper presents a self-supervised method for learning reliable visual correspondence from unlabeled videos. We formulate the correspondence as finding paths in a joint space-time graph, where nodes are grid patches sampled from frames, and are linked by two types of edges. Our learned representation outperforms the state-of-the-art self-supervised methods on a variety of visual tasks.
  arXiv  Detail & Related papers  (2021-09-28T05:40:01Z)
• Beyond Single Instance Multi-view Unsupervised Representation Learning [21.449132256091662]
  We impose more accurate instance discrimination capability by measuring the joint similarity between two randomly sampled instances. We believe that learning joint similarity helps to improve the performance when encoded features are distributed more evenly in the latent space.
  arXiv  Detail & Related papers  (2020-11-26T15:43:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.