Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation

• URL: http://arxiv.org/abs/2305.07283v3
• Date: Thu, 31 Aug 2023 03:47:34 GMT
• Title: Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation
• Authors: Zewen Zheng, Guoheng Huang, Xiaochen Yuan, Chi-Man Pun, Hongrui Liu, and Wing-Kuen Ling
• Abstract summary: Few-shot segmentation (FSS) aims to segment unseen classes given only a few samples. We introduce a quaternion perspective on correlation learning and propose a novel Quaternion-valued Correlation Learning Network (QCLNet) Our QCLNet is formulated as a hyper-complex valued network and represents correlation tensors in the quaternion domain, which uses quaternion-valued convolution to explore the external relations of query subspace.
• Score: 33.88445464404075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot segmentation (FSS) aims to segment unseen classes given only a few annotated samples. Encouraging progress has been made for FSS by leveraging semantic features learned from base classes with sufficient training samples to represent novel classes. The correlation-based methods lack the ability to consider interaction of the two subspace matching scores due to the inherent nature of the real-valued 2D convolutions. In this paper, we introduce a quaternion perspective on correlation learning and propose a novel Quaternion-valued Correlation Learning Network (QCLNet), with the aim to alleviate the computational burden of high-dimensional correlation tensor and explore internal latent interaction between query and support images by leveraging operations defined by the established quaternion algebra. Specifically, our QCLNet is formulated as a hyper-complex valued network and represents correlation tensors in the quaternion domain, which uses quaternion-valued convolution to explore the external relations of query subspace when considering the hidden relationship of the support sub-dimension in the quaternion space. Extensive experiments on the PASCAL-5i and COCO-20i datasets demonstrate that our method outperforms the existing state-of-the-art methods effectively. Our code is available at https://github.com/zwzheng98/QCLNet and our article "Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation" was published in IEEE Transactions on Circuits and Systems for Video Technology, vol. 33,no.5,pp.2102-2115,May 2023,doi: 10.1109/TCSVT.2022.3223150.

Related papers

• Anchor-aware Deep Metric Learning for Audio-visual Retrieval [11.675472891647255]
  Metric learning aims at capturing the underlying data structure and enhancing the performance of tasks like audio-visual cross-modal retrieval (AV-CMR) Recent works employ sampling methods to select impactful data points from the embedding space during training. However, the model training fails to fully explore the space due to the scarcity of training data points. We propose an innovative Anchor-aware Deep Metric Learning (AADML) method to address this challenge.
  arXiv  Detail & Related papers  (2024-04-21T22:44:44Z)
• Time Series Compression using Quaternion Valued Neural Networks and Quaternion Backpropagation [3.8750364147156247]
  We propose a novel quaternionic time-series compression methodology where we divide a long time-series into segments of data. We extract the min, max, mean and standard deviation of these chunks as representative features and encapsulate them in a quaternion. This time-series is processed using quaternion valued neural network layers.
  arXiv  Detail & Related papers  (2024-03-18T12:22:11Z)
• Spatial-Temporal Decoupling Contrastive Learning for Skeleton-based Human Action Recognition [10.403751563214113]
  STD-CL is a framework to obtain discriminative and semantically distinct representations from the sequences. STD-CL achieves solid improvements on NTU60, NTU120, and NW-UCLA benchmarks.
  arXiv  Detail & Related papers  (2023-12-23T02:54:41Z)
• Efficient Semantic Matching with Hypercolumn Correlation [58.92933923647451]
  HCCNet is an efficient yet effective semantic matching method. It exploits the full potential of multi-scale correlation maps. It eschews the reliance on expensive match-wise relationship mining on the 4D correlation map.
  arXiv  Detail & Related papers  (2023-11-07T20:40:07Z)
• FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network [48.912196729711624]
  Few-shot semantic segmentation is the task of learning to locate each pixel of a novel class in a query image with only a few annotated support images. We propose a Feature-Enhanced Context-Aware Network (FECANet) to suppress the matching noise caused by inter-class local similarity. In addition, we propose a novel correlation reconstruction module that encodes extra correspondence relations between foreground and background and multi-scale context semantic features.
  arXiv  Detail & Related papers  (2023-01-19T16:31:13Z)
• An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning [77.72330187258498]
  We propose a novel Entropy-guided Reinforced Partial Convolutional Network (ERPCNet) ERPCNet extracts and aggregates localities based on semantic relevance and visual correlations without human-annotated regions. It not only discovers global-cooperative localities dynamically but also converges faster for policy gradient optimization.
  arXiv  Detail & Related papers  (2021-11-03T11:13:13Z)
• Hypercorrelation Squeeze for Few-Shot Segmentation [33.45698767126036]
  Few-shot semantic segmentation aims at learning to segment a target object from a query image using only a few annotated support images of the target class. This challenge requires to understand diverse levels of visual cues and analyze fine-grained correspondence relations between the query and the support images. We propose Hypercorrelation Squeeze Networks (HSNet) that leverages multi-level feature correlation and efficient 4D convolutions.
  arXiv  Detail & Related papers  (2021-04-04T05:27:13Z)
• A Quaternion-Valued Variational Autoencoder [15.153617649974263]
  variational autoencoders (VAEs) have proved their ability in modeling a generative process by learning a latent representation of the input. We propose a novel VAE defined in the quaternion domain, which exploits the properties of quaternion algebra to improve performance.
  arXiv  Detail & Related papers  (2020-10-22T12:33:42Z)
• Pairwise Similarity Knowledge Transfer for Weakly Supervised Object Localization [53.99850033746663]
  We study the problem of learning localization model on target classes with weakly supervised image labels. In this work, we argue that learning only an objectness function is a weak form of knowledge transfer. Experiments on the COCO and ILSVRC 2013 detection datasets show that the performance of the localization model improves significantly with the inclusion of pairwise similarity function.
  arXiv  Detail & Related papers  (2020-03-18T17:53:33Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.